Support Generic Storage Layer For Jobs? (MongoDB)

mvniekerk / tokio-cron-scheduler

Schedule tasks on Tokio using cron-like annotation

Apache License 2.0

521 stars 60 forks source link

Support Generic Storage Layer For Jobs? (MongoDB) #54

Open LuisOsta opened 1 year ago

LuisOsta commented 1 year ago

We currently use MongoDB as a primary database for one of our services that aim to use this library. And we were hoping to be able to use MongoDB as the storage for jobs (the way y'all currently support Postgres and NATs).

I had two requests/ideas:

What would be the work involved in getting native MongoDB support?
Would it be possible to support a generic storage trait that could be implemented by the end user so that they could use whatever storage layer they wanted?

If possible/needed I'd be happy to help contribute to getting either/both of those done

mvniekerk commented 1 year ago

Hi Luis The async MongoDB seems quite fine to do this with. https://docs.rs/mongodb/latest/mongodb/ There are generic traits - one for in-memory, Nats and Postgres. If you need any assistance let me know.

LuisOsta commented 1 year ago

@mvniekerk My main question would be about how to integrate the mongodb client with the rest of the library the way we have for Nats, Postgres, etc. Any guidance on that?

LuisOsta commented 1 year ago

@mvniekerk For instance I have tried to create a custom implementation of the Metadata and Notification Store but there's a lot of things (like DataStore) that aren't externally accessible and so a custom implementation outside of the crate can't be done. The postgres and nats implementation have special access due to being internal to the crate

LuisOsta commented 1 year ago

Also does this library support having multiple services try to interact with the same job queue? (So that you don't have multiple services handling the same job)?

mvniekerk commented 2 months ago

Hi @LuisOsta It has been a while for this report. Have you come right?