search

film42 / sidekiq-rs

A port of sidekiq to rust using tokio
MIT License
95 stars 10 forks source link

Namespaces #7

Closed film42 closed 1 year ago

film42 commented 1 year ago

Ruby uses redis-namespace to partition a single redis store among multiple apps using the default queue. Technically namespaces are an anti-pattern because it's now possible to create many different queues; however, many companies (including the one I work for) is using redis-namespace with sidekiq. It's certainly a "if it ain't broke, don't fix it thing" so we haven't. Others too.

I think we can add this in two places:

  1. Make this lib aware of namespaces and use a namespaced key for every redis operation. This works but it's a bit clunky.
  2. Add support to rust redis for namespacing, or put a shim around the redis connection to be namespaced. This feels a bit heavy handed.

It would be easier to implement (1) but (2) would be preferred.

film42 commented 1 year ago

I think the best solution is going to be a custom redis connection wrapper. I can reimplement the bb8_redis crate in the sidekiq lib to wrap the common commands used and set a namespace for each key. This may cause more allocations but it should be fairly swappable. I have a pretty broken branch with this implemented, and I'll have a PR up soon. cc @justmark

justmark commented 1 year ago

Thats fantastic. You have the only library for Sidekiq in Rust that supports Tokio + consumers. Can't wait to take it for a spin :)

film42 commented 1 year ago

This has been released, along with a bb8 upgrade, a new namespaced example, and a bunch more docs in the README in version 0.6.0.