31 July 2009 ~ 6 Comments

Bootstrapping from Colombia: Developed on Google App Engine

Lenguajero is running on Google App Engine (GAE). This started as an experiment. I wanted to see how fast I could get it up and running.  The answer, as it turns out, is very fast. Any developer can get up and running and through the initial tutorial in a Sunday afternoon. We’re still running within the free-limits set by GAE, and as we start to get more and more traffic to our site I’ll blog about the operational costs.

Python & Django

I’ve launched websites using Ruby on Rails/MySQL and Java/Struts/MySQL, but until I started using GAE I had never coded with Python or Django. When we started working on Lenguajero GAE was only offered in Python. It is now available in Java. Given the choice I would still choose Python for this project.

As far as Python goes it’s OK, and I love to hate it. It’s like an ugly child who gets good grades, you know you should love them for who they are, but…  Some small things that drive me crazy and lead to ugly code include:

  • str(var) instead of var.str()
  • no ternary operator – you’re really going to ask me to thank you for that??!!
  • var = i, (including the hanging-comma) creates a tuple??!! This has tripped me up more than once when refactoring dict assignment code. See automated testing

And then there’s Django… We’re currently only using Django for templating and our internationalization. It is bundled into GAE, although it’s an old version (0.96). Django feels very “it’s-for-your-own-good” restrictive (there is no elsif). I know there are developers out there who love this. I am not one of them.

Helpful Google App Engine Infrastructure

Right out-of-the-box GAE provides an infrastructure that I found incredibly helpful in developing Lenguajero.  They include:

  • a development server (that has never crashed)
  • deployment management with versioning
  • an administration console that tracks (almost) realtime
    • resource load
    • regex log parsing
    • graphical performance monitoring

I had all these tools at my disposal at Amazon. The console especially gives me a warm-and-fuzzy feeling. This is the stuff that lets you understand and respond to operational issues. As a development-team of 1, not having to set this up myself is important.

With Google App Engine I can look at scaling as a monetary issue instead of a development-time issue. I have a lot of cached data using Google’s Memcache API that add very little complexity to the code. The site *should* survive a slashdot effect (¡ojalá!) without any further development, but it might get expensive!

I am a strong believer in automated unit and functional testing. Even though this is a 1-developer project I believe having extensive test coverage saves more time than it costs. Especially when you’re coding with a duck-typed language.

There is no official GAE documentation to set up an automated-testing framework (this is a gaping hole). The following helped me immensely:
Unit Test Your Google App Engine Models
Using Fixture To Test A Google App Engine Site
Testing Applications with WebTest

Before you use Google App Engine wrap your head around this.
It took me about a month to really get using an object-based DB and not having full SQL capabilities.

  • No table JOINs
  • No ORs
  • Duplicate data instead of using an OR. For example, Lenguajero has a Conversation class:
    class Conversation(db.Model):
         from_member = db.ReferenceProperty(Member, collection_name="from_member_reference_set")
         to_member = db.ReferenceProperty(Member, collection_name="to_member_reference_set")

    A requirement emerges that we want to display to a user all of their previous conversations. This can’t be done using a single query in GQL on the Conversation table. Instead this information is duplicated in a new class:

    class MemberConversation(db.Model):
         member = db.ReferenceProperty(Member)
         conversation = db.ReferenceProperty(Conversation)
  • No SQL UPPER, LOWER or LIKE – I don’t want to allow both usernames “hotstuff1979″ and “HotStuff1979″. So instead the Member class looks like this:
    class Member(db.Model):
         username = db.StringProperty(required=True)
         lowercase_username = db.StringProperty(required=True)

    Lookup is done on lowercase_username.

Weird things about Lenguajero because of GAE

  • You’d expect that GAE would have some sort of search capabilities, because, well, it’s Google. But no. There is no wildcard search or LIKE operator. This means that Lenguajero Language Exchange Partner Search has no wildcard support for usernames. Here is an illuminating thread on the subject.

All in all working on top of GAE has been a good experience. I was able to get started with Python, Django and GAE and build an entire website from scratch. Having the infrastructure ready to go has let me focus on writing code and working on the stuff that really matters to our users. If you have scarce resources (eg. 1 developer) and want to be up and running in a short time-frame, GAE will cut your development time while allowing you to scale. As we scale our traffic and expand our features I’ll keep you updated on how Lenguajero performs on Google App Engine.

6 Responses to “Bootstrapping from Colombia: Developed on Google App Engine”

  1. decastro 12 January 2010 at 4:48 am Permalink

    Many people are enthu about its functionality even if they haven’t used it

    http://bygsoft.wordpress.com/2010/01/09/cloudy-combo-google-app-engine-and-amazon-s3-combo-pack/


Leave a Reply