Support Generic Storage Layer For Jobs? (MongoDB)

mvniekerk / tokio-cron-scheduler

Schedule tasks on Tokio using cron-like annotation

Apache License 2.0

453 stars 54 forks source link

Support Generic Storage Layer For Jobs? (MongoDB) #54

Open LuisOsta opened 10 months ago

LuisOsta commented 10 months ago

We currently use MongoDB as a primary database for one of our services that aim to use this library. And we were hoping to be able to use MongoDB as the storage for jobs (the way y'all currently support Postgres and NATs).

I had two requests/ideas:

What would be the work involved in getting native MongoDB support?
Would it be possible to support a generic storage trait that could be implemented by the end user so that they could use whatever storage layer they wanted?

If possible/needed I'd be happy to help contribute to getting either/both of those done

mvniekerk commented 10 months ago

Hi Luis The async MongoDB seems quite fine to do this with. https://docs.rs/mongodb/latest/mongodb/ There are generic traits - one for in-memory, Nats and Postgres. If you need any assistance let me know.

LuisOsta commented 10 months ago

@mvniekerk My main question would be about how to integrate the mongodb client with the rest of the library the way we have for Nats, Postgres, etc. Any guidance on that?

LuisOsta commented 10 months ago

@mvniekerk For instance I have tried to create a custom implementation of the Metadata and Notification Store but there's a lot of things (like DataStore) that aren't externally accessible and so a custom implementation outside of the crate can't be done. The postgres and nats implementation have special access due to being internal to the crate

LuisOsta commented 10 months ago

Also does this library support having multiple services try to interact with the same job queue? (So that you don't have multiple services handling the same job)?