Clustered Redis Slots Refresh Ignores TLS Usage

aballman commented 1 year ago

Describe the problem When connecting to a TLS enabled cluster (specifically AWS Elasticache, but doesn't seem relevant), there is a timeout when refreshing cluster slots due to TLS not being enabled.

ERRO[0017] Cluster refresh failed: redisc: all nodes failed
read tcp xxx.xxx.xxx.xxx:50221->xxx.xxx.xxx.xxx:6379: i/o timeout

In the gomodule/redigo library, which is a dependency of mna/redisc there are two pathways to connect that are being utilized by redis_exporter Dial and DialURL

DialURL is used when connecting not in cluster mode, and Dial is used when clustering is enabled.

Clustering Enabled: https://github.com/oliver006/redis_exporter/blob/de4f5eb0bfc3e15e3aa882db262572ed3bb45dcb/exporter/redis.go#LL98C10-L98C10

Clustering Disabled: https://github.com/oliver006/redis_exporter/blob/de4f5eb0bfc3e15e3aa882db262572ed3bb45dcb/exporter/redis.go#LL53

The problem here is that in the redigo lib when using DialURL it automatically appends the DialUseTLS DialOption when the scheme is set to rediss. This does not happen in Dial https://github.com/gomodule/redigo/blob/d6854479365f0307560fa28e18e2bd0634b05229/redis/conn.go#L389

What version of redis_exporter are you running? Please run redis_exporter --version if you're not sure what version you're running. 1.48.0

Running the exporter What's the full command you're using to run the exporter? (please remove passwords and other sensitive data)

REDIS_PASSWORD=<redacted>
REDIS_EXPORTER_IS_CLUSTER=true
REDIS_ADDR=<redacted>
REDIS_EXPORTER_SKIP_TLS_VERIFICATION=true
./redis_exporter

Expected behavior Exporter does not time out when attempting to refresh cache slots on a TLS enabled cluster.

Screenshots N/A

Additional context N/A

aballman commented 1 year ago

oliver006 commented 1 year ago

This is a great find!

aballman commented 1 year ago

oliver006 / redis_exporter

Clustered Redis Slots Refresh Ignores TLS Usage #780