Open oguzhannkayaa opened 4 years ago
wow no upd over here. What a nice project...
I know this is old but hope it would help
the key is to set redbeat_redis_url to sentinel dsn
like for example, your sentinel dsn might look like this:
sentinel://:redis@redis-sentinel:26379/0;sentinel://:redis@redis-sentinel2:26379/0;sentinel://:redis@redis-sentinel3:26379/0
but redbeat/scheduler.py:124 checks for
elif conf.redis_url.startswith('redis-sentinel') and 'sentinels' in redis_options:
...
so you have to change your sentinel dsn (for redbeat only) to reidis-sentinel
please note that it is perfectly valid of celeryconfig
to use sentinel
as a schema within the dsn but not for redbeat tho
anyways, if your sentinel requires auth - provide password
to sentinel_kwargs
the other options are for redis connection only
my celeryconfig.py
looks like that
# at this point we would just dump it from celery_settings (pydantic model)
locals().update(celery_settings.model_dump())
# now we can override some
if redis_settings.redis_mode == "sentinel":
broker_url = redis_settings.sentinel_dsn
result_backend_transport_options = REDIS_PARAMS
broker_transport_options = REDIS_PARAMS
redbeat_redis_url = redis_settings.sentinel_dsn.replace("sentinel://", "redis-sentinel://")
redbeat_redis_options = {
"sentinels": redis_settings.sentinels,
"sentinel_kwargs": {},
"password": redis_settings.password,
"db": redis_settings.db,
"service_name": redis_settings.master_name,
"retry_on_timeout": True,
"socket_timeout": 1,
"socket_connect_timeout": 1,
"socket_keepalive": True,
"socket_keepalive_options": {socket.TCP_KEEPCNT: 5, socket.TCP_KEEPIDLE: 60, socket.TCP_KEEPINTVL: 10},
"retry_period": 10,
}
debug output would look like that
Configuration ->
. broker -> sentinel://:**@redis-sentinel:26379/0
. loader -> celery.loaders.default.Loader
. scheduler -> redbeat.schedulers.RedBeatScheduler
. redis -> redis-sentinel://:**@redis-sentinel:26379/0;redis-sentinel://:redis@redis-sentinel2:26379/0;redis-sentinel://:redis@redis-sentinel3:26379/0
. lock -> `redbeat::lock` 25.00 minutes (1500s)
. logfile -> [stderr]@%WARNING
. maxinterval -> 5.00 minutes (300s)
Hi, I am trying to add multiple redis (for high availability) url to redbeat_redis_url and I also wants to add broker_url for rabbitmq.
I am trying to use rabbitmq as a message broker. and redis as a beat data store. because of clustering.
app.conf.update( broker_url = 'amqp://localhost:5672/', redbeat_redis_url = 'redis://master1:6379/1', redbeat_redis_options = { 'sentinels': [('slave2:6379'), ('slave3.18:6379')], 'socket_timeout': 0.1, 'retry_period': 0.1
} ) After I close master, beat stop. Does go to red_beat_redis_options
and also I am getting this error when I run celery beat -S redbeat.RedBeatScheduler.
beat raised exception : ConnectionError('Error -2 connecting to redis-sentinel:26379. Name or service not known.',)
How can I create a service_name and password in redis-sentinel I am not trying to use redis as a message broker. I am using celery-redbeat to store celerybeat data in redis-sentinel cluster from this page.https://pypi.org/project/celery-redbeat/
and
from this configuration
redbeat_redis_url = 'redis-sentinel://redis-sentinel:26379/0' redbeat_redis_options = { 'sentinels': [('192.168.1.1', 26379), ('192.168.1.2', 26379), ('192.168.1.3', 26379)], 'socket_timeout': 0.1, }
I add 192.168.1.1:26379 instead of redis-sentinel:/26379 but when master node down in redis-sentinel cluster beat is down too.
redbeat_redis_url = 'redis-sentinel://192.168.1.1:26379/0' redbeat_redis_options = { 'sentinels': [('192.168.1.2', 26379), ('192.168.1.3', 26379)], 'socket_timeout': 0.1, }