Closed vicpara closed 1 year ago
Ok. It seems the entire configuration for Redis Elasticache is not suitable for using with BullMQ as we may loose data due to the default and unchangeable eviction policy. It all makes sense. Closing this.
You can configure the eviction policy in Elasticache, in fact it works pretty well with BullMQ, I have used it myself for years.
I tried in the latest version and it tells me changing eviction property isn't possible.
I think the issue is not that it cannot be changed, but that they do not allow BullMQ to check the correct maxmemory policy: https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/ParameterGroups.Redis.html#ParameterGroups.Redis.4-0-10
TLDR; if you know that the eviction policy is correct, you can ignore the warning.
@manast, sorry for the dumb questions, but is the eviction policy log correct at all? The official redis docs say this:
The policies volatile-lru, volatile-lfu, volatile-random, and volatile-ttl behave like noeviction if there are no keys to evict matching the prerequisites.
In other words, if no keys with expire
set to true exist, nothing will be evicted. My newb assumption is that Bullmq has control over whether expire
is set to true.
@JohnPeberdyRR locks on jobs have expire set, so they could be randomly removed by Redis if choosing any of those policies.
@manast, thank you for clarifying.
Version
4.12.1
Platform
NodeJS
What happened?
I'm trying to connect BullMQ from a node AWS ECS task to Redis Elasticache. I'm getting the following errors right at the service starts when I'm very sure there is no data being pushed right away in the queues. So the DB is not under pressure.:
IMPORTANT! Eviction policy is volatile-lru. It should be "noeviction" IMPORTANT! Eviction policy is volatile-lru. It should be "noeviction" IMPORTANT! Eviction policy is volatile-lru. It should be "noeviction"
How to reproduce.
Can it be that the errors are caused by a too small Redis instance? The Redis instance i try to deploy to has 0.5Gb ram.
Relevant log output
No response
Code of Conduct