search

film42 / sidekiq-rs

A port of sidekiq to rust using tokio
MIT License
95 stars 10 forks source link

Redis #35

Closed zigomi closed 7 months ago

zigomi commented 7 months ago

For this project, will Redis have an advantage over, let's say, Postgres or MySql?

Is Redis here due to the fact that the original, in Ruby, SideKiq uses it?

film42 commented 7 months ago

Hey @zigomi . One of the goals of this crate is to be able to interoperate with ruby sidekiq apps. In order to do that, redis needs to be used.

Technically, other systems can be used, but they need to support a few features like:

  1. An IO.select or long-poll operation to check a queue (work queues).
  2. A priority style queue with access (time based queues for crons and future work).
  3. Data structures for reporting metrics to power the sidekiq-web metrics. That whole ruby project would need to be rewritten.
  4. Good built-in expiry for keys (self cleaning).

While technically possible, it's beyond the scope of this crate. I've come across other crates in the past that specialize background workers with postgres or disk backed storage. I can't seem to remember those names. They're out there!