Grist has needed a job queue for some time. This adds one, using BullMQ. BullMQ however requires Redis, meaning we couldn't use jobs for the large subset of Grist that needs to be runnable without Redis (e.g. for use on desktop, or on simple self-hosted sites). So simple immediate, delayed, and repeated jobs are supported also in a crude single-process form when Redis is not available.
This code isn't ready for actual use since an important issue remains to be worked out, specifically how to handle draining the queue during deployments to avoid mixing versions (or - if allowing mixed versions - thinking through any extra support needed for the developer to avoid introducing hard-to-test code paths).
Has this been tested?
[x] 👍 yes, I added tests to the test suite
[ ] 💭 no, because this PR is a draft and still needs work
Context
Grist has needed a job queue for some time. This adds one, using BullMQ. BullMQ however requires Redis, meaning we couldn't use jobs for the large subset of Grist that needs to be runnable without Redis (e.g. for use on desktop, or on simple self-hosted sites). So simple immediate, delayed, and repeated jobs are supported also in a crude single-process form when Redis is not available.
This code isn't ready for actual use since an important issue remains to be worked out, specifically how to handle draining the queue during deployments to avoid mixing versions (or - if allowing mixed versions - thinking through any extra support needed for the developer to avoid introducing hard-to-test code paths).
Has this been tested?