Monday, January 26, 2009

Google App Engine argh

I have written and hosted my latest site: PixPlz using Google App Engine. I don't really recommend GAE to anyone unless you really, really value using a free service.

Here's a list of minor limitations you might run into:
  • no aggregate queries
  • no image handling
  • hard to port
  • index building
  • bizarre errors from simple commands
  • must sort on inequality filter first
  • random 1000 limits
no aggregate queries
This one is pretty frustrating if you ever like to gather statistics about a single damn thing your app is storing. I would like a total of the number of times all my images have been voted on, but if I try to loop through every image (which appears to be the only way to do it) I quickly run into Google's 5 second page timeout limit.

no image handling
I know that technically, there's an image library, but it doesn't do a damn thing if you actually want to handle images. Say someone uploads an image: want to know the filetype? Read the individual bytes to determine it. Want to know the dimensions? Yeah, read the bytes. You can do a few write-only changes like resizing, but you can't read any attributes about the image you're manipulating.

hard to port
Other people have written about this, but eventually I'll want to get away from GAE, and it gets harder and harder the more code I write.

index building
Want to query some data in a new way? Cool, just write the query, test it on your dev machine and push to production. Oh, wait, you have to sit and stare at a progress indicator for an indeterminate amount of time before you can actually use that query. This took over 5 minutes for 100 photos:
bizarre errors from simple commands
When I got up to about 150 photos, I started getting timeout errors from this GQL query:
photos = db.GqlQuery('SELECT * FROM Photo')
photos.count()

Yes, counting 150 photos is difficult for BigTable to accomplish, and results in an timeout error about 1% of the time.

must sort on inequality filter first
I wanted to make a page that showed the top rated girls, but not one that had maybe one or two "yes" votes, and were thus at 100% falsely. Simple, right? You just query for all the photos that have more than 5 votes and sort them by score. But wait, you have to sort by votes first, then score, giving you a useless top 10 board of mostly the oldest photos (because they have the most votes) thanks BigTable!

random 1000 limits
Can't return more than 1000 elements from a query, can't offset more than 1000 elements into a result set, can't handle uploads or any variables in memory bigger than 1MB. Google touts how well App Engine scales, but I can make any system scale if I just don't let you do anything with it.

4 comments:

Anonymous said...

*Laughs all the way to his cushy relational database*

Joel said...

Yes. Whenever you see the words "App" and "Engine" in the same phrase, run away. I recommend Django instead!

Ari Wilson said...

Sounds like you need to spend more time thinking about your schema.

Blake Householder said...

Sounds like you need to spend more time thinking about how to leave a useful comment on a blog.