PostgreSQL > MySQL (Facebook MySQL Article Response)

I saw this original article via LinkedIn and a conversation ensued on Google+ and I thought some of the comments were worth posting here.

The original article: http://gigaom.com/cloud/facebook-trapped-in-mysql-fate-worse-than-death/

The question was raised, that because PostgreSQL was an old SQL model, wont it have the same problems as MySQL and Wes Devauld (of BeefyApps fame) commented stating:

Wes Devauld: Postgres was built from the ground up using MVCC, which has a much smaller overhead in update transactions. MySQL InnoDB 5.1 added MVCC, but I’ve read that the implementation is half assed and they are still hamstrung on how they handle the transaction logs.

I thought it was a comment worth preserving, for posterity.

Tags: , , ,

Xen Cloud Platform; Mount Disk in different Virtual Host

Here at BeefyApps we run our infrastructure atop the Xen Cloud Platform to maximize the utility of physical hardware. Working in a virtual environment has some excellent advantages for when things do not go quite as planned. Say for instance while messing with a boot loader, or kernel you ended up with a machine that was not in a bootable state. Because all the disks are on the hypervisor, we can just take the device and mount it on another virtual machine without needing a screwdriver.

xe vm-list
xe vm-disk-list vm=<dead-vm>
xe vdb-create vm-uuid=<good-vm> device=1 vdi-uuid=<disk-vdi-from-dead-vm> mode=RW type=Disk
# returned newly-minted-uuid
xe vbd-plug uuid=<newly-minted-uuid>

First we find our <dead-vm> and <good-vm> UUIDs and keep them handy. Look up the VDI (not the VDB) from the dead host; depending on your setup, you may have a lot of disks listed for the machine, in our case we only had the system default Local storage.

With the virtual device interface we create a new block device on a good host, and plug it in. Our example uses device 1, which will create and install /dev/xvdb in the virtual machine. Using device 0 gives you /dev/xvda which is probably already in use. Inside the good host we simply mapped the partitions and mounted them

kpartx -a /dev/xvdb
mount /dev/mapper/xvdb3 /mnt
Tags: , , , , ,

Export iCal to Excel or CSV

I had what I thought was a simple task. I wanted to get a group calendar that I was putting together in Google out into a list that I could print out to pass around to the *ahem* more tactilely oriented among us.

Google Calendar does print and agenda format which is okay except for the fact that it adds in the person who created the event and the calendar that the event came from. This is great if you want to know all the information, but if you are just trying to print off a quick list for people to read, they often don’t want to look through multiple pages of repetitive text. Also, think of the tree-folk.

So I thought, I have desktop programs that can read a calendar, surely they will have a view that corresponds to my needs. I looked at iCalendar for OSX and I happened to have a copy of Entourage (first time using it, I guess it is like Outlook light?), but neither of these applications met my needs. Entourage teased me on to the point of getting me to a list that I thought looked good, but then would not let me print it.

So then I thought to myself, “self”, I said, because I always refer to myself in the third person, the second person sounds so accusatory, “you’re a programmer, how hard could this problem be?”

So I took a look at the Ruby libraries and I immediately found http://icalendar.rubyforge.org/. So I installed it

gem install icalendar

and created this little ruby program to export a CSV for my easy printing needs.

#!/usr/bin/env ruby
 
require 'rubygems'
require 'icalendar'
require 'date'
 
ARGV.each do |filename|
  file = File.open(filename)
  cals = Icalendar.parse(file)
 
  cals.each do |calendar|
    calendar.events.each do |event|
      puts "\"" + event.dtstart.strftime("%Y-%m-%d %H:%M") + "\",\"" + event.dtend.strftime("%Y-%m-%d %H:%M") + "\",\"" + event.summary + "\",\"" + event.location + "\""
    end
  end
end

I hope this helps someone out there. I was surprised how quickly I managed to get that up and running, it took me longer to write the blog post than it did to write the code.

Tags: , , , , ,

WordPress PostgreSQL Trouble – PHP Fatal error: Cannot redeclare class wpdb

So over here at BeefyApps are using PostgreSQL instead of MySQL for our WordPress backend. There is this discussion in the WordPress codex which talks about why they do not natively support PostgreSQL out of the box.

However, we have decided that PostgreSQL is the way forward for us so we needed to find a work around. Luckily for us there is a plugin, PostgreSQL for WordPress, which will allow us to do this.

So I followed the steps that were laid out in the Readme.txt that is bundled with the package and I eventually came to run the installation (after the configuration) and I was presented with a blank white screen. Not so lucky after all…

I went through the logs and found:

PHP Fatal error:  Cannot redeclare class wpdb

Which was originating in the class wp-db.php in the wp-includes directory. This was being invoked from the wp-admin/setup-config.php file.

After some searching on the internet I came across some other posts talking about modifying the wp-functions.php file and some others which didn’t really seem appropriate because they would be overwritten with an upgrade. I did find that in the wp-admin/install.php file, there was a reference to wp-db.php as part of the installer. I commented this section out and the installer ran successfully.

I think this problem occurs because before the db.php file can run and eval the class definition inside the wp-db.php file, after it makes it’s replacements for the file, the install.php file has already loaded and processed that class into memory. When we try to redefine the class it throws an error. Anyway it seems to work now.

Tags: ,

Connecting to a Xen Console

So earlier I had one solution for connecting to a Xen domU instance that you had created, but I just came across another, more native way and I thought I should include it up here.

Once you have created your domU guest you can find it’s domain id by entering:

xe vm-list params=dom-id uuid=<uuid-of-vm> --minimal

That will give you the domain id and then you just plug that into:

/usr/lib/xen/bin/xenconsole <dom-id>

and you are off to the races.

I would like to credit the XVP installation guide for pointing that out to me:
http://www.xvpsource.org/?topic=build&page=appliancevm

Tags: , ,

UITextView Scrolling Doesn’t Work

I had an issue where I was unable to get scrolling on a UITextView to work when editing was turned off. I searched and searched through the vast sea of information that is the internet and came up empty.

Basically the problem was that I had nested my UITextView inside of a UIImageView. This seems innocuous enough but UIImageView does not have the userInteractionEnabled flag set to YES by default. Therefore the event was dispatched up to that view from the main thread and was blocked because the view would not handle and propagate the event.

So the solution was to set the UIImageView’s userInteractionEnabled flag to YES.

    self.userInteractionEnabled = YES;

I think there could be other views that are set up like this as well, although I am not sure which ones they would be. So watch out for this, especially if you are defining your views in code.

I hope that this shows up in someone’s Google search and helps them out.

Tags: , , , , ,

Custom Fonts in iOS 4

Up till now there hasn’t been an easy way to add custom fonts to your iPhone applications.  As of iOS 4 it has become very easy to do.  Here is what you need to do in order to add custom fonts:

  1. Add your custom font files into your project using XCode as a resource
  2. Add a key to your info.plist file called UIAppFonts.
  3. Make this key an array
  4. For each font you have, enter the full name of your font file (including the extension) as items to the UIAppFonts array
  5. Save info.plist
  6. Now in your application you can simply call [UIFont fontWithName:@"CustomFontName" size:12] to get the custom font to use with your UILabels and UITextViews, etc…

It’s that simple!

Tags: , , , ,

CoreData SQLite3 Database Constraint Failed Error

Have you ever seen the following error message:

Error Domain=NSCocoaErrorDomain Code=256 UserInfo=0x3baae90 "Operation could not be completed. (Cocoa error 256.)

Talk about not Helpful.

Inside this there was a userInfo dictionary with the following message:

error during SQL execution : constraint failed

Slightly more helpful, but not by very much.

We came across this error message while we were developing our Electrophobe game. We started the debugging process by inserting logging code and pinpointing where the code was failing out. After about 6 hours of work combing through both code and rebuilt databases we finally found the solution.

The trick was that the primary key store for the SQLite3 database that was backing our CoreData NSPersistenceStore had an incorrect value for the Z_MAX column. This column is used, we believe, to house the maximum primary key in order to quickly retrieve it for additional inserts.

As is evident from our escapade however, it is clearly a little vulnerable to tampering when loading the data directly into the SQLite database. To fix the issue, simply update the table with the max(Z_ID) from the referenced table and it should all hook up again.

Tags: , , , , ,

Posting Multiple Files With ASI

ASI (from All-Seeing Interactive) is a great product.  Apple has made strides to incorporate some of the advanced procedures that this allows for, but ASI gives you some great features to easily handle some more complicated operations. It will allow you to easily download multiple files at once and aggregate the progress monitoring of these; It allowed for upload progress monitoring, before apple wired up its NSConnection objects for this purpose and many other features. Having said that I have run across a couple of scenarios and extensions to the basic code that I have found useful.

First I had the need to post multiple files in the same post request up to a web server. The ASI stack has a basic method for attaching an NSData object to your request data. This method simply takes in the data and assigns it “filename” as the default filename for the data that is passed in. Realistically there is no way for ASIFormDataRequest to infer the filename from the binary data. However, this can be quite useful when you are posting binary file data to a web service as the web service is often looking for unique filename identification of each binary part of the posted multipart-form request.

In order to enhance the ASIFormDataRequest to handle this scenario we simply modified the setData: forKey: method to have an override that sets the filename property.

?View Code OBJECTIVE-C
- (void)setData:(NSData *)data forKey:(NSString *)key
{
	[self setData:data forKey:key withFilename:@"file"];
}
 
- (void)setData:(NSData *)data forKey:(NSString *)key withFilename:(NSString *)filename
{
	if (!fileData) {
		fileData = [[NSMutableDictionary alloc] init];
	}
	NSMutableDictionary *file = [[[NSMutableDictionary alloc] init] autorelease];
	[file setObject:data forKey:@"data"];
	[file setObject:filename forKey:@"filename"];
	[fileData setValue:file forKey:key];
	[self setRequestMethod:@"POST"];
}

By adding the override on the method we have maintained the core functionality and extended it to allow for multiple files with multiple file names.

Tags: , , , , ,

Simple Ruby OAuth with Twitter

We’ve been working on creating a website that would access resources within the cloud, and decided on adopting OAuth as an enabler. The difficulty was in finding a concise and simple example on how to do just that. Everywhere people had fully integrated rails models, controllers and libraries, or complicated explanations with gaps in the code provided.

To fix the problem, we’ve come up with a very simple OAuth example using the Twitter Provider. You’ll need the oauth gem for this to work:

#!/usr/bin/env ruby
 
require 'rubygems'
require 'oauth'
 
consumer = OAuth::Consumer.new "twitter_provided_consumer_key",
                     "twitter_provided_secret_key",
                      { :site => 'http://twitter.com/',
                        :request_token_path => '/oauth/request_token',
                        :access_token_path => '/oauth/access_token',
                        :authorize_path => '/oauth/authorize'}
 
request_token = consumer.get_request_token
puts "Place \"#{request_token.authorize_url}\" in your browser"
print "Enter the number they give you: "
pin = STDIN.readline.chomp
 
access_token = request_token.get_access_token(:oauth_verifier => pin)
 
puts access_token.get('/account/verify_credentials.json')

As you can see, the OAuth gem deals with most of the dirty work, as a matter of fact, almost half of the example file is environment, dependencies and user interaction. There is really only three lines that have any real use.

Tags: , , , , ,