search

taskforcesh / bullmq

BullMQ - Message Queue and Batch processing for NodeJS and Python based on Redis
https://bullmq.io
MIT License
6.15k stars 404 forks source link

Bug on specfied the `keyPrefix` on redis option. #1219

Open cnwangjie opened 2 years ago

cnwangjie commented 2 years ago

I specified the keyPrefix to redis option. Then whenever I specify the prefix or not on bullmq option it will not work.

I found a similar issue for that (#560). I can understand that I need to provide the same prefix option on the Queue and the Worker, but what about on I provide keyPrefix on redis option?

manast commented 2 years ago

You should not use ioredis prefix as it is making some magic with keys and BullMQ handles prefixes on its own. Maybe we should throw an exception if this setting is set.

cnwangjie commented 2 years ago

Hi, @manast

Thank you for your reply. Whether sharing the Redis client instance with others' usage is by design or not? If not, why bullmq allowed to pass a redis instance directly?

manast commented 2 years ago

@cnwangjie so that you can save Redis connections, some users have a limited amount of connections to use.

cnwangjie commented 2 years ago

@manast Yes, I also hope to save connections. Even use one Redis instance for multiple projects. It would be great if Redis keyPrefix option could be supported.

I guess we can get options from redis client instance for a simple implementation.

manast commented 2 years ago

I also wished we could support it, but unfortunately, it is very very complicated to do, sorry. Best we can do is throw an error, if the keyPrefix is used by mistake.

Rush commented 2 years ago

Should users simply prefix queue names new Queue(prefix + '_jobs') if two applications need to use the same redis instance?

manast commented 2 years ago

You can use different queue names or use the prefix option: https://api.docs.bullmq.io/interfaces/QueueOptions.html#prefix

Rush commented 2 years ago

You can use different queue names or use the prefix option: https://api.docs.bullmq.io/interfaces/QueueOptions.html#prefix

Thanks. I presume there's no way to globally provide this?

manast commented 2 years ago

No, you need to specify it in every Queue and Worker instance.

skirsten commented 2 months ago

Best we can do is throw an error, if the keyPrefix is used by mistake.

Please add that :pray: also for the connection option, I just spend more than a hour tracking down why the jobs were getting stalled when they should not. And it was because of the keyPrefix :face_exhaling: