AppEngine quota frequently exceeded

By the afternoon, Caliper is unable to publish results due to exceeding the 
quota limits. By not showing the results to the console, this makes it 
unusable. Hopefully by being a Google project, Caliper can obtain an exemption 
and have an increased free quota.

Tips
----
I suspect that the AppEngine application is naive, e.g. making frequent data 
store calls, which are extremely costly. Instead you can batch writes by having 
a cron task flush a queue, e.g. in memcached. If the potential lossiness of 
memcache is unacceptable, Blobstore is a workaround. It have a storage quota 
but not read/write quota, letting it be used as a scratchpad of temporary 
results. The cron task can accumulate a set, delete, and update the data store. 
The races can be masked by having the results in memcached, with a rare case of 
being evicted while still not in the datastore.

One might even go as far as assuming that a single user never publishes 
benchmarks in parallel. The datastore could be avoided entirely by having a 
blob file per user and rewriting it. That would avoid the quota limits.

Another problematic use of AppEngine is that it is overly expensive to the 
quota when hosting websites. Instead it can act as an API tier and not serve 
content. An extremely cheap / free hosting site can serve the content and make 
api calls to fetch the results.

Basically, the free tier should be more than enough but you have to design 
around avoiding AppEngine quota issues.

Original issue reported on code.google.com by Ben.Manes@gmail.com on 21 Jul 2013 at 1:02

Merged into: #255

ccristian / caliper

AppEngine quota frequently exceeded #267