Closed ghost closed 1 year ago
@bootc, do you know anyone who can help us know what probes are these in the logs that gets 500 responses when the slaves are not found? It starts to get annoying when they go down while you use them.
redis.sentinel.SlaveNotFoundError: No slave found for 'redis'
127.0.0.6 - - [16/Feb/2023:07:51:29 +0000] "GET /login/ HTTP/1.1" 500 1605 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:51:39 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:51:49 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:51:59 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:52:09 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:52:19 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:52:29 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:52:39 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:52:49 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:52:59 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
redis.sentinel.SlaveNotFoundError: No slave found for 'redis'
127.0.0.6 - - [16/Feb/2023:07:53:09 +0000] "GET /login/ HTTP/1.1" 500 1605 "-" "kube-probe/1.22+"
redis.sentinel.SlaveNotFoundError: No slave found for 'redis'
127.0.0.6 - - [16/Feb/2023:07:53:19 +0000] "GET /login/ HTTP/1.1" 500 1605 "-" "kube-probe/1.22+"
redis.sentinel.SlaveNotFoundError: No slave found for 'redis'
127.0.0.6 - - [16/Feb/2023:07:53:29 +0000] "GET /login/ HTTP/1.1" 500 1605 "-" "kube-probe/1.22+"
redis.sentinel.SlaveNotFoundError: No slave found for 'redis'
127.0.0.6 - - [16/Feb/2023:07:53:29 +0000] "GET /login/ HTTP/1.1" 500 1605 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:53:39 +0000] "GET /login/ HTTP/1.1" 500 1605 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:53:49 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:53:59 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:54:09 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
127.0.0.6 - - [16/Feb/2023:07:54:19 +0000] "GET /login/ HTTP/1.1" 200 5120 "-" "kube-probe/1.22+"
Hi!
I am creating a new issue for this since none of the old articles here solved my issue. I deployed netbox using
bootc/netbox-cart
with external serversbitnami/redis
(repo) &bitnami/postgresql
(repo) all running in the same namespace. The deployment is working but every 4-5mins an error pops up in one of the pods that saysnetbox redis.sentinel.SlaveNotFoundError: No slave found for ‘redis’
. We get the page below when it happens.On kubectl events, we get the following:
Any ideas on what to check? or how to fix this?
Additional info for the netbox pods: Logs:
PS: sentinel & redis authentication is already disabled. Here are the settings for the
values.yaml
file onbootc/netbox-chart
:bitnami/redis
's currentvalues.yaml
file:In addition, netbox-worker pod is also losing the it's connection to redis-master when the master is moved to another pod. it cannot find the master anymore after it moved and continuously says,
No master found
Here are the logs:the following are the logs for redis nodes, redis-node-0:
redis-node-1