Add queue-based job processing

[skluck]

Currently the agent requires cron jobs for processing builds and releases.

"Pending" jobs are read from the database by the agent it will immediately start all pending jobs. We need a better way to process jobs, a method that uses queues and allows throttling or capacity management.

Eventually I'd like the agent to not need direct db access so that is something to consider.

[skluck]

We'll update the agent to process jobs using the AWS Beanstalk Worker pattern:

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html

How it works:

1. Worker listens for POST http requests.
2. Worker processes jobs and Beanstalk takes messages off the queue if it responds with 200.

   If the worker responds with any other response code, the message will be retried or sent to dlq based on settings

Changes required

1. Add an index.php sqsd-handler in the agent codebase which parses the message and starts the appropriate CLI command to process the job.

   We'll need a local web-server for this, the built-in php web-server may be good enough

2. Add sqsd equivalent for local development.

   For handling the queue - there are a few open-source equivalents since amz's sqsd is not open source. (https://github.com/gilbitron/sqsd)

3. The frontend will need to add messages to the queue for processing.

---

For now this is the quickest way to switch from naive cronjobs and allow the agents to scale up.

Long term we don't want to require aws so we'll need to make sure anything dealing with queues is able to be abstracted away (so we can use beanstalkd or other options).