redis / ioredis

🚀 A robust, performance-focused, and full-featured Redis client for Node.js.
MIT License
14.37k stars 1.2k forks source link

ssl3_get_record:wrong version number, when connecting to sentinel via tls #1204

Closed Tobjoern closed 3 years ago

Tobjoern commented 4 years ago

Hey, I have a docker swarm, which sets up redis sentinel, with 3 sentinels, 1 master and one slave. I have now added tls (config files included). Now I want to connect to the sentinel clusters via ioredis, but I keep receiving the following error in the master's logs: "Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number". So there is obviously an ssl error occuring, but I don't know why. The master and slave connect correctly (via tls), and the sentinels correctly connect to the master and slave, there are no errors in the logs. But I cannot connect to the sentinel instances via tls. I should also note, that I had no problems connecting with ioredis, when I hadn't enabled tls. I'm just really confused, it could be that I misconfigured redis or sentinel, but as there is no issue with them connecting to each other, I assume that I don't use ioredis properly. Moreover I actually am able to connect briefly ('connect' event fires), but immediately after that the 'error' event fires, suggesting that there is a problem with the key exchange.

Your help is highly appreciated.

Here is the relevant code:

const client = new redis({
    sentinels: [
        {
            host: 'redis-sentinel1',
            port: 26380,
        },
        {
            host: 'redis-sentinel2',
            port: 26380,
        },
        {
            host: 'redis-sentinel3',
            port: 26380,
        }
    ],
    name: "mymaster",
    password: redisPassword,
    enableTLSForSentinelMode: true,
    sentinelTLS: {
        ca: [caCert],
        key: tlsKey,
        cert: tlsCert,
        secureProtocol: 'TLSv1_2_method'
    },
}
)

redis.conf

daemonize no
masterauth %%REDIS_PASSWORD%%
requirepass %%REDIS_PASSWORD%%
replica-announce-ip %%SLAVE_ANNOUNCE_IP%%
tls-port 6379
port 0
tls-auth-clients yes
tls-ca-cert-file %%TLS_CA_CERT_FILE%%
tls-key-file %%TLS_KEY_FILE%%
tls-cert-file %%TLS_CERT_FILE%%
tls-replication yes
tls-protocols "TLSv1.2 TLSv1.3"

sentinel.conf

dir /tmp
sentinel monitor mymaster %%SENTINEL_MASTER_IP%% 6379 %%SENTINEL_QUORUM%%
sentinel down-after-milliseconds mymaster %%SENTINEL_DOWN_AFTER%%
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster %%SENTINEL_FAILOVER%%
sentinel auth-pass mymaster %%REDIS_PASSWORD%%
port 0
tls-auth-clients yes
tls-replication yes
tls-ca-cert-file %%TLS_CA_CERT_FILE%%
tls-key-file %%TLS_KEY_FILE%%
tls-cert-file %%TLS_CERT_FILE%%
tls-port 26380
tls-protocols "TLSv1.2 TLSv1.3"
Jav3k commented 3 years ago

I also faced the same problem. But it was solved with the addition of the tls field (duplicate of sentinelTLS):

const sentinels = [
  { host: '127.0.0.1', port: 26380 },
  { host: '127.0.0.1', port: 26379 },
];

const tlsSettings = {
  ca: [fs.readFileSync('../../tls/ca.crt')],
  cert: fs.readFileSync('../../tls/client.crt'),
  key: fs.readFileSync('../../tls/client.key'),
  rejectUnauthorized: false,
};

const redisClient = new Redis({
  sentinels,
  password: '123123',
  sentinelPassword: '123123',
  name: 'my-master',
  role: 'master',
  tls: tlsSettings,
  sentinelTLS: tlsSettings,
  enableTLSForSentinelMode: true,
});

I have same configuration of redis.conf and sentinel.conf. node 12, ioredis 4.19.4

I think it's because of this: https://github.com/luin/ioredis/blob/c2f634fcc1ed25a47d368e96e432012db99efa34/lib/connectors/SentinelConnector/index.ts#L132

ctrlaltdylan commented 3 years ago

Bumping this because I'm using Heroku to create a throng'd process worker.js that uses Bull to process background jobs.

I've noticed that upgrading the heroku redis instance causes this same error.

The only fix is to totally delete the instance and starting from scratch.

Still experimenting.

ctrlaltdylan commented 3 years ago

Ended up being Heroku using self signed certs with v6 Redis

kennydifiore commented 3 years ago

@ctrlaltdylan I'm also using Heroku Redis and receiving this error. What should be done, if anything, to resolve it?

ctrlaltdylan commented 3 years ago

@kennydifiore My solution is to optionally add a flag that will disable security checks for redis:

let options = {
  port: REDIS_PORT,
  host: REDIS_HOST,
  password: REDIS_PASSWORD,
};

if (process.env.REDIS_INSECURE_TLS) {
  options.tls = {
    rejectUnauthorized: false,
  };
}

Step 2: stop using Heroku. This is a blatant security hole they're not fixing and they refuse to offer refunds for the downtime you incur & this security risk to you.