recommended redis.conf for rrq

[wlandau]

I am beginning a rewrite of crew, and part of my plan is to build on top of rrq and take responsibility for configuring and launching short-lived instances of redis-server (initiate when a pipeline begins, terminate when the pipeline ends or crashes). I am writing to ask your advice on a good redis.conf file. What settings would you suggest to only enable the Redis commands that rrq uses?

I read through https://raw.githubusercontent.com/redis/redis/7.0/redis.conf, and so far I am thinking about the following template for ephemeral conf files. (I do not know how to use the TLS settings, and I do not know where to find the certificates or how to securely deliver them to clients, but it would be nice to have eventually.)

# set dynamically
bind {{insert_local_IP_on_the_LAN_or_VPC}} # 127.0.0.1 -::1
port {{insert_random_ephemeral_port}} # 6379
# requirepass {{insert_random_long_password}}
proc-title-template "{{insert_name}} {title} {port} {tls-port}"
loglevel {{insert_loglevel}} # notice

# logs
set-proc-title yes
logfile ""

# scale
daemonize no
cluster-enabled no
timeout 0
tcp-keepalive 300
maxclients 10000
databases 1
# io-threads 4
# io-threads-do-reads no

# storage
save ""
appendonly no
appendfsync no
shutdown-timeout 0
shutdown-on-sigint nosave
shutdown-on-sigterm nosave

# security
protected-mode yes
enable-protected-configs no
enable-debug-command no
enable-module-command no

# tls
# tls-port 6379
# tls-cert-file redis.crt
# tls-key-file redis.key
# tls-key-file-pass secret
# tls-cert-file redis.crt
# tls-key-file redis.key
# tls-key-file-pass secret
# tls-client-cert-file client.crt
# tls-client-key-file client.key
# tls-client-key-file-pass secret
# tls-ca-cert-file ca.crt
# tls-ca-cert-dir /etc/ssl/certs
# tls-auth-clients no
# tls-auth-clients optional
# tls-protocols "TLSv1.2 TLSv1.3"
# tls-ciphers DEFAULT:!MEDIUM
# tls-ciphersuites TLS_CHACHA20_POLY1305_SHA256
# tls-prefer-server-ciphers yes

[richfitz]

redux does not support tls (yet, there's a long-standing PR but it needs work and as I don't use TLS it's not been a priority).

We always use redis as a container, so just the default configuration really. It's possible that other configurations will provide higher performance but I expect that would only be an issue if you were doing a great many very small jobs.

There are security issues with exposing redis to the internet at large, and I believe that the password will only partly mitigate this as an attacker can try many passwords a second to try and brute force it. So it's more important to think about the overall security of the communication. If you're using ssh tunnels between the client computer and your Redis instance on AWS (or similar) then you should be fine. We typically do all communication between Redis and the rrq queue/workers on a docker private network.

I am not sure what the penalty is for a maxclients that high, but it seems unlikely that you would be able to keep that many R processes happy.

[wlandau]

Thanks for the advice, Rich. I agree, the password system in Redis is weak as password systems go. And from https://redis.io/docs/management/security/, I do understand that I will probably need to put the Redis server on the same network as the workers, with a tunnel or API for user-server communication. Still, I am not sure I understand what exactly makes SSH more secure than Redis + TLS, given that SSH also relies on TLS for security.