search

cooperaj / laravel-redis-sentinel

Provides a Redis driver for Laravel that is Sentinel aware
16 stars 7 forks source link

Daemon notification #2

Open msencenb opened 7 years ago

msencenb commented 7 years ago

First of all, thanks for creating this. I just deployed it to my production laravel stack and it's working great so far.

In my HA setup I normally run supervisor with laravel queues in daemon mode to keep things running at a lower CPU level. Unfortunately it seems like daemon based queues aren't able to renegotiate the redis connection when the sentinel master goes down. When I run the queues in non daemon mode, it appears to renegotiate the connection successfully.

I'm totally cool running my queues in non daemon mode, and if things started to affect performance I'll just spin up a dedicated queue box. But if you run your queues in daemon mode I would love to hear about how you got it working.

Thanks!

cooperaj commented 7 years ago

We've been seeing this issue sadly and are working on it. It seems to typically result in an exception storm into the logs.

netdata-be commented 7 years ago

@cooperaj Do you already have found something where I can help you with? We are hitting the same bug.

jbardnz commented 6 years ago

Hey guys,

I know this is a very old issue but I have found half a solution to this. I am using a scheduled task that runs every minute - it loops through each redis instance in the sentinal and checks if it is the master. If the master is different from the last time it ran then it restarts the queues with Artisan::call('queue:restart');. I have a gist with my script here: https://gist.github.com/jbardnz/75fe228ebbb1a8fac6c14d8ca8a1e49d

Note that it specifically uses the file cache to save the current master just incase redis is having trouble (hence the master/slave swap).

I hope that might help some of you :)