Major refactor of the entire app + switch to node-resque

[dmfenton]

Added

• New status of Cached when something is in the db
• New class Cache handles all interaction with the koop cache
• New class Work handles dispatch of all jobs
• New class CSVQueue handles logic for putting csv requests on to the queue
• New class CSVRequest handles getting and storing a csv
• New class FeatureRequest handles putting a job on the queue or requesting features locally
• New class Portal handles all communication with an instance of Portal
• New class ImportService handles the actual job of importing a service
• Queue Lock plugin adapted from Node-Resque's standard to prevent duplicate jobs from getting on the queue
• New queue management routes get agol/queue/length, get agol/queue/workers, get agol/queue/jobs, get /agol/queue/clearFailed

  Changed

• Simplified Caching logic flow
• Workers now use node-resque multiworker
• Workers now started by running node worker.js from the project root
• Requests for a resource without a file extension return info only
• All exports go through workers if they are enabled
• Export file name is saved to the info doc at insert time
• Now sending 500 if geohash on AWS is empty
• Internal queues for preventing duplicate csv and feature request jobs replaced with QueueLock and writing lock files to "disk"

  Fixed

• Logic is now consistent for creating export file names

[ungoldman]

y'all got some rebasin' to do

[dmfenton]

Rebased :)

[ungoldman]

There's a whole lot going on in here (~11000 lines added/deleted) in one big commit. Should I just merge and hope for the best?

[chelm]

This should be converted to es2015 first

[ungoldman]

:neutral_face:

[dmfenton]

Not sure how I could have split this up, everything is kinda interrelated. Given that its been through pretty substantial testing locally and is going to go through a lot of QA before release I think we'll be pretty safe when we release it.

That said, I would love some feedback on the way I structured this thing.

@chelm ha, I think I'll save that for after the honeymoon.

[dmfenton]

I'm going merge this and publish a pre-release. That will make it easier for us to take this through our formal QA. We can still incorporate any bug fixes and feedback before a final release.

[ungoldman]

:+1: