grantcarthew / node-rethinkdb-job-queue

A persistent job or task queue backed by RethinkDB.
https://github.com/grantcarthew/node-rethinkdb-job-queue/wiki
MIT License
157 stars 16 forks source link

List of improvements #91

Open grantcarthew opened 4 years ago

grantcarthew commented 4 years ago

Improvements for the queue

I am working on a new job queue backed by MongoDB.

This issue is to list items that could improve rethinkdb-job-queue that pop into mind whilst working on the new queue.

I don't plan on implementing these changes, they are for someone else who want's to upgrade this package.

Move DB Connection Function

Move the database connection from the constructor to a queue.connect() method. This allows to user to receive the returned promise and respond to errors.

Retry the DB Connection

If the database becomes disconnected, instead of crashing, go into a pause state and retry the connection periodically.

Prevent Exceptions

This is a module within a larger application. It should not bring down the application if something goes wrong. Add options to log rather than throw exceptions.

Place Job Data into a Job.data Property

Change the way jobs are created to put the user data into an extra property such as data. This separates job attributes from work to be done.

grantcarthew commented 4 years ago

I'm scrapping the idea of building a job queue on MongoDB. It is such a bad database for something like this.

For reference: https://blog.meteor.com/mongodb-queries-dont-always-return-all-matching-documents-654b6594a827

I'll leave this here for now.