Recommend default queuing backend for Active Job

[ellery-nava]

What's the task?

Active Job comes out of the box with an in-memory queueing system. However, this is not sufficient for a production system - because the queue only exists in memory, pending tasks will be lost if the system restarts.

This ticket is to investigate the pros and cons of various queuing systems and recommend a default backend that will work well for most new Nava projects.

[ellery-nava]

@rocketnova @lorenyu @SammySteiner - starting to take a look at some of the recommended libraries from the documentation.... I assume that backends that require Redis or another queue service not currently a part of the infrastructure would be less attractive than something like Good Job that runs using Postgres?

Any thoughts on recommending a single library, vs. doing a short writeup on pros/cons of a couple options? Wondering whether something like Resque, for example, might become more attractive if a project is already planning on using Redis.

[rocketnova]

@ellery-nava I think a short writeup on pros/cons sounds very helpful, especially in terms of considerations that teams should account for when selecting a backend.

Licensing and hosting costs should definitely be a consideration. I'd say that it's possible for something that is Postgres-backed to be more expensive than something Redis-backed, but I think it really depends. From a quick search, it sounds like ElastiCache for Redis is cheaper but really best used as a caching layer, rather than as the primary data store, whereas MemoryDB for Redis is great as the primary data store, but really expensive.

[lorenyu]

if needed SQS-backed is also ok. Azure has an equivalent.

Postgres-backed would be easiest since that wouldn't be different between AWS and Azure