Specs for v1

[jgallen23]

Goals

• process long running jobs
• queue up after reaching a max concurrent jobs
• be able to spin up multiple workers
• get progress of job
• jobs can log process
• mongodb backed
• be able to retry failed jobs
• if app crashes, restart jobs in progress
• future jobs - example: run a job at midnight, or run in 24 hours from some event
• json API - maybe we just do this in a hapi plugin that uses this lib
• job timeouts - job is marked fail after X seconds

  Similar Libs

• https://github.com/rschmukler/agenda - focused on future jobs
• https://github.com/learnboost/kue - uses redis

  Use Cases

• sending email
• aggregating metrics
• maintenance tasks
• drip marketing campaigns
• crunching numbers

[jgallen23]

lets do a little research to see if there are other libs out there that we could just use. I think kue is the best, but I'd prefer not to have to add redis as a dependency to most of our apps.

[dawnerd]

https://github.com/scttnlsn/monq - Nice and basic. https://github.com/qualiancy/kinetik - Looks pretty cool. Supports priority too.

Found some others but they're redis based and no where near as good as kue is.

I really like the way kinetik is laid out. Has pretty much everything we'd need.

[jgallen23]

after looking through them, would you recommend us using an existing one or building our own?

[dawnerd]

Tough one... For me its between agenda and kinetik. If all those goals are important then we'd be better of building something custom. Otherwise I'd say agenda on the pure fact that it's been updated more often and supports forking.

[jgallen23]

going to go with agenda. turning this repo into hapi-agenda. closing this