grahamneville
commented
9 months ago I'm also hitting this issue. Are there any updates as to why this might be occurring?
Open sohamM97 opened 1 year ago
grahamneville
commented
9 months ago I'm also hitting this issue. Are there any updates as to why this might be occurring?
chunechew
commented
1 month ago @sohamM97 @grahamneville
I've written a temporary monkey-patching codes for fixing this bug:
The temporary code (e.g. path/to/sentinel.py):
from urllib.parse import urlparse
from django.core.exceptions import ImproperlyConfigured
from django.utils.module_loading import import_string
from django_redis.pool import ConnectionFactory
class CustomSentinelConnectionFactory(ConnectionFactory):
def __init__(self, options):
# allow overriding the default SentinelConnectionPool class
pool_cls_path = options.setdefault(
"CONNECTION_POOL_CLASS", "redis.sentinel.SentinelConnectionPool"
)
self.pool_cls = import_string(pool_cls_path)
self.pool_cls_kwargs = options.get("CONNECTION_POOL_KWARGS", {})
sentinel_cls_path = options.get("SENTINEL_CLASS", "redis.sentinel.Sentinel")
self.sentinel_cls = import_string(sentinel_cls_path)
redis_client_cls_path = options.get("REDIS_CLIENT_CLASS", "redis.client.Redis")
self.redis_client_cls = import_string(redis_client_cls_path)
connection_pool_cls_path = options.get(
"CONNECTION_POOL_CLASS", "redis.sentinel.SentinelConnectionPool"
)
self.connection_pool_cls = import_string(connection_pool_cls_path)
self.redis_client_cls_kwargs = options.get("REDIS_CLIENT_KWARGS", {})
self.options = options
sentinels = options.get("SENTINELS")
if not sentinels:
raise ImproperlyConfigured(
"SENTINELS must be provided as a list of (host, port)."
)
self.min_other_sentinels = options.get("MIN_OTHER_SENTINELS", 0)
self.sentinel_kwargs = options.get("SENTINEL_KWARGS", {})
# provide the connection pool kwargs to the sentinel in case it
# needs to use the socket options for the sentinels themselves
connection_kwargs = self.make_connection_params(None)
connection_kwargs.pop("url")
connection_kwargs.update(self.pool_cls_kwargs)
self.connection_kwargs = connection_kwargs
self._sentinel = self.sentinel_cls(
sentinels,
self.min_other_sentinels,
self.sentinel_kwargs,
**self.connection_kwargs,
)
def get_connection_pool(self, params):
url = urlparse(params["url"])
master_name = url.hostname
if master_name is None:
raise ValueError(
'SENTINEL_SETTINGS["master_name"] must be specified in the Django settings.'
)
pool = self.connection_pool_cls(
master_name, self._sentinel, **self.connection_kwargs
)
return pool
An example of the Django settings file (settings.py):
import ssl
# ...
REDIS_SSL_SETTINGS = {
"ssl_ca_certs": "relative/path/to/redis_ssl_ca_certs.crt", # Modify this to the correct file name and path
"ssl_certfile": "relative/path/to/redis_ssl_certfile.crt", # Modify this to the correct file name and path
"ssl_keyfile": "relative/path/to/redis_ssl_keyfile.key", # Modify this to the correct file name and path
}
DJANGO_REDIS_CONNECTION_FACTORY = "path.to.sentinel.CustomSentinelConnectionFactory" # Modify this to the correct file name and path
SENTINEL_HOST_NAME = "sentinel.host.name" # Modify this to the correct Sentinel host name
SENTINEL_PORT = 26379 # Modify this to the correct Sentinel port number
SENTINELS = [
(SENTINEL_HOST_NAME , SENTINEL_PORT),
]
REDIS_PASSWORD = "password" # Modify this to the correct password
SENTINEL_MASTER_NAME = "mymaster" # Modify this to the correct name of the master
REDIS_PORT = 6379 # Modify this to the correct port number of the master and the slaves
REDIS_DB = 0 # Modify this to the correct DB number of the master and the slaves
SENTINEL_KWARGS = {
"password": REDIS_PASSWORD,
"ssl": True,
"ssl_cert_reqs": ssl.CERT_REQUIRED,
**REDIS_SSL_SETTINGS,
}
SENTINEL_SETTINGS = {
"master_name": SENTINEL_MASTER_NAME,
"sentinel_kwargs": SENTINEL_KWARGS,
}
REDIS_SENTINEL_COMMON_KWARGS = { # You may change or add the values
"max_connections": 100,
"ssl": True, # Don't change or remove this
"db": REDIS_DB
"port": REDIS_PORT,
"ssl_cert_reqs": ssl.CERT_REQUIRED,
**REDIS_SSL_SETTINGS, # Don't change or remove this
}
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": f"rediss://{SENTINEL_MASTER_NAME}/{REDIS_DB}",
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.SentinelClient",
"CONNECTION_POOL_KWARGS": REDIS_SENTINEL_COMMON_KWARGS,
"REDIS_CLIENT_KWARGS": REDIS_SENTINEL_COMMON_KWARGS,
"PASSWORD": REDIS_PASSWORD,
"SENTINELS": SENTINELS,
"SENTINEL_KWARGS": SENTINEL_KWARGS,
"CONNECTION_POOL_CLASS": "redis.sentinel.SentinelConnectionPool",
},
}
}
# ...Until the Django-Redis dev team fixes the bug, we can avoid it using these.
Describe the bug We are trying to use django redis cache with sentinel and SSL connection in our application. The issue occurs when we use
redissas part of the URL given in theLOCATIONsetting, as described below.To Reproduce
Keep the CACHES setting on settings.py as follows:
Note that we take the settings such as REDIS_MASTER, REDIS_DB etc from our environment file (.env), which we read using the django-environ library.
Expected behavior The library should be able to connect to the sentinel-based setup and the application should start up properly on running the server.
Stack trace
Environment (please complete the following information):
Additional context Instead of keeping the LOCATION as
f"{REDIS_PROTOCOL}://{REDIS_MASTER}/{REDIS_DB}", if we keep it asf"redis://{REDIS_MASTER}/{REDIS_DB}", the application works as expected. We are confused as to why we need to keep the protocol asrediseven though we clearly want to userediss.