search

OT-CONTAINER-KIT / redis-operator

A golang based redis operator that will make/oversee Redis standalone/cluster/replication/sentinel mode setup on top of the Kubernetes.
https://ot-redis-operator.netlify.app/
Apache License 2.0
731 stars 207 forks source link

Sentinel and replication are in one chart. #949

Closed dxygit1 closed 1 month ago

dxygit1 commented 1 month ago

Sentinel and replication are in the same chart, but Sentinel isn't being launched. However, when they are in separate charts, they launch normally. I want Sentinel and replication to be executed within the same chart.

What version of redis operator are you using? 0.17.0

kubectl logs <_redis-operator_pod_name> -n <namespace>
{"level":"info","ts":"2024-05-23T10:27:22Z","logger":"setup","msg":"starting manager"}
{"level":"info","ts":"2024-05-23T10:27:22Z","msg":"starting server","kind":"health probe","addr":"[::]:8081"}
{"level":"info","ts":"2024-05-23T10:27:22Z","logger":"controller-runtime.metrics","msg":"Starting metrics server"}
{"level":"info","ts":"2024-05-23T10:27:22Z","logger":"controller-runtime.metrics","msg":"Serving metrics server","bindAddress":":8080","secure":false}
I0523 10:27:22.432436       1 leaderelection.go:250] attempting to acquire leader lease redis1/6cab913b.redis.opstreelabs.in...
I0523 10:27:37.560080       1 leaderelection.go:260] successfully acquired lease redis1/6cab913b.redis.opstreelabs.in
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting EventSource","controller":"redis","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"Redis","source":"kind source: *v1beta2.Redis"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting Controller","controller":"redis","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"Redis"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting EventSource","controller":"redissentinel","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisSentinel","source":"kind source: *v1beta2.RedisSentinel"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting EventSource","controller":"redisreplication","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisReplication","source":"kind source: *v1beta2.RedisReplication"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting Controller","controller":"redisreplication","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisReplication"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting EventSource","controller":"redissentinel","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisSentinel","source":"kind source: *v1beta2.RedisReplication"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting Controller","controller":"redissentinel","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisSentinel"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting EventSource","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster","source":"kind source: *v1beta2.RedisCluster"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting Controller","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster"}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting workers","controller":"rediscluster","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisCluster","worker count":1}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting workers","controller":"redissentinel","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisSentinel","worker count":1}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting workers","controller":"redis","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"Redis","worker count":1}
{"level":"info","ts":"2024-05-23T10:27:37Z","msg":"Starting workers","controller":"redisreplication","controllerGroup":"redis.redis.opstreelabs.in","controllerKind":"RedisReplication","worker count":1}
{"level":"info","ts":"2024-05-23T10:27:50Z","logger":"controllers.RedisReplication","msg":"Reconciling opstree redis replication controller","Request.Namespace":"redis1","Request.Name":"redis-replication"}
{"level":"info","ts":"2024-05-23T10:27:50Z","logger":"controllers.RedisSentinel","msg":"Reconciling opstree redis controller","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:27:50Z","logger":"controllers.RedisSentinel","msg":"Redis Replication is specified but not ready, so will reconcile again in 10 seconds","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:27:50Z","logger":"controllers.RedisReplication","msg":"Redis replication nodes are not ready yet","Request.Namespace":"redis1","Request.Name":"redis-replication","Ready.Replicas":"0","Expected.Replicas":3}
{"level":"info","ts":"2024-05-23T10:27:50Z","logger":"controllers.RedisReplication","msg":"Reconciling opstree redis replication controller","Request.Namespace":"redis1","Request.Name":"redis-replication"}
{"level":"info","ts":"2024-05-23T10:27:50Z","logger":"controllers.RedisReplication","msg":"Redis replication nodes are not ready yet","Request.Namespace":"redis1","Request.Name":"redis-replication","Ready.Replicas":"0","Expected.Replicas":3}
{"level":"info","ts":"2024-05-23T10:28:00Z","logger":"controllers.RedisSentinel","msg":"Reconciling opstree redis controller","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:00Z","logger":"controllers.RedisSentinel","msg":"Redis Replication is specified but not ready, so will reconcile again in 10 seconds","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:10Z","logger":"controllers.RedisSentinel","msg":"Reconciling opstree redis controller","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:10Z","logger":"controllers.RedisSentinel","msg":"Redis Replication is specified but not ready, so will reconcile again in 10 seconds","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:20Z","logger":"controllers.RedisSentinel","msg":"Reconciling opstree redis controller","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:20Z","logger":"controllers.RedisSentinel","msg":"Redis Replication is specified but not ready, so will reconcile again in 10 seconds","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:30Z","logger":"controllers.RedisSentinel","msg":"Reconciling opstree redis controller","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:30Z","logger":"controllers.RedisSentinel","msg":"Redis Replication is specified but not ready, so will reconcile again in 10 seconds","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:40Z","logger":"controllers.RedisSentinel","msg":"Reconciling opstree redis controller","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}
{"level":"info","ts":"2024-05-23T10:28:40Z","logger":"controllers.RedisSentinel","msg":"Redis Replication is specified but not ready, so will reconcile again in 10 seconds","Request.Namespace":"redis1","Request.Name":"redis-sentinel"}

image

image

drivebyer commented 1 month ago

execute info replication on all redis node, show the result plz

dxygit1 commented 1 month ago

info

[root@172-16-120-6 redis-replication]# kubectl  -n redis1 logs redis-replication-0 
Redis is running without password which is not recommended
Setting up redis in standalone mode
Running without persistence mode
Running without TLS mode
ACL_MODE is not true, skipping ACL file modification
Starting redis service in standalone mode.....
1:C 23 May 2024 10:27:34.147 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 23 May 2024 10:27:34.147 * Redis version=7.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 23 May 2024 10:27:34.147 * Configuration loaded
1:M 23 May 2024 10:27:34.148 * monotonic clock: POSIX clock_gettime
1:M 23 May 2024 10:27:34.149 # Failed to write PID file: Permission denied
1:M 23 May 2024 10:27:34.149 * Running mode=standalone, port=6379.
1:M 23 May 2024 10:27:34.149 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 23 May 2024 10:27:34.149 * Server initialized
1:M 23 May 2024 10:27:34.149 * Ready to accept connections tcp
1:M 23 May 2024 10:28:34.151 * Replica 100.89.15.96:6379 asks for synchronization
1:M 23 May 2024 10:28:34.151 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for '24fa167fcc015ecd8c8bc91782e72df52fec9a0d', my replication IDs are '34ee7c68450f711cd18dcd347c958f9b2518d963' and '0000000000000000000000000000000000000000')
1:M 23 May 2024 10:28:34.151 * Replication backlog created, my new replication IDs are 'd61d13fbf77bc324bc452135f19d025d3f93354c' and '0000000000000000000000000000000000000000'
1:M 23 May 2024 10:28:34.151 * Delay next BGSAVE for diskless SYNC
1:M 23 May 2024 10:28:34.350 * Replica 100.119.38.217:6379 asks for synchronization
1:M 23 May 2024 10:28:34.350 * Partial resynchronization not accepted: Replication ID mismatch (Replica asked for '8ca1fdc42523e7198e3b846d77d9770f2361c0cb', my replication IDs are 'd61d13fbf77bc324bc452135f19d025d3f93354c' and '0000000000000000000000000000000000000000')
1:M 23 May 2024 10:28:34.350 * Delay next BGSAVE for diskless SYNC
1:M 23 May 2024 10:28:39.327 * Starting BGSAVE for SYNC with target: replicas sockets
1:M 23 May 2024 10:28:39.327 * Background RDB transfer started by pid 104
104:C 23 May 2024 10:28:39.328 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
1:M 23 May 2024 10:28:39.328 * Diskless rdb transfer, done reading from pipe, 2 replicas still up.
1:M 23 May 2024 10:28:39.334 * Background RDB transfer terminated with success
1:M 23 May 2024 10:28:39.335 * Streamed RDB transfer with replica 100.89.15.96:6379 succeeded (socket). Waiting for REPLCONF ACK from replica to enable streaming
1:M 23 May 2024 10:28:39.335 * Synchronization with replica 100.89.15.96:6379 succeeded
1:M 23 May 2024 10:28:39.335 * Streamed RDB transfer with replica 100.119.38.217:6379 succeeded (socket). Waiting for REPLCONF ACK from replica to enable streaming
1:M 23 May 2024 10:28:39.335 * Synchronization with replica 100.119.38.217:6379 succeeded
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# kubectl  -n redis1 logs redis-replication-1
Redis is running without password which is not recommended
Setting up redis in standalone mode
Running without persistence mode
Running without TLS mode
ACL_MODE is not true, skipping ACL file modification
Starting redis service in standalone mode.....
1:C 23 May 2024 10:27:54.717 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 23 May 2024 10:27:54.717 * Redis version=7.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 23 May 2024 10:27:54.717 * Configuration loaded
1:M 23 May 2024 10:27:54.717 * monotonic clock: POSIX clock_gettime
1:M 23 May 2024 10:27:54.718 # Failed to write PID file: Permission denied
1:M 23 May 2024 10:27:54.718 * Running mode=standalone, port=6379.
1:M 23 May 2024 10:27:54.718 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 23 May 2024 10:27:54.719 * Server initialized
1:M 23 May 2024 10:27:54.719 * Ready to accept connections tcp
1:S 23 May 2024 10:28:52.485 * Before turning into a replica, using my own master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
1:S 23 May 2024 10:28:52.485 * Connecting to MASTER 100.86.3.100:6379
1:S 23 May 2024 10:28:52.485 * MASTER <-> REPLICA sync started
1:S 23 May 2024 10:28:52.485 * REPLICAOF 100.86.3.100:6379 enabled (user request from 'id=17 addr=100.119.38.215:60218 laddr=100.89.15.96:6379 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=46 qbuf-free=20428 argv-mem=23 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=37807 events=r cmd=slaveof user=default redir=-1 resp=3 lib-name=go-redis(,go1.21.10) lib-ver=9.5.1')
1:S 23 May 2024 10:28:52.486 * Non blocking connect for SYNC fired the event.
1:S 23 May 2024 10:28:52.486 * Master replied to PING, replication can continue...
1:S 23 May 2024 10:28:52.486 * Trying a partial resynchronization (request 24fa167fcc015ecd8c8bc91782e72df52fec9a0d:1).
1:S 23 May 2024 10:28:57.663 * Full resync from master: d61d13fbf77bc324bc452135f19d025d3f93354c:14
1:S 23 May 2024 10:28:57.664 * MASTER <-> REPLICA sync: receiving streamed RDB from master with EOF to disk
1:S 23 May 2024 10:28:57.666 * Discarding previously cached master state.
1:S 23 May 2024 10:28:57.666 * MASTER <-> REPLICA sync: Flushing old data
1:S 23 May 2024 10:28:57.666 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 23 May 2024 10:28:57.670 * Loading RDB produced by version 7.2.3
1:S 23 May 2024 10:28:57.670 * RDB age 18 seconds
1:S 23 May 2024 10:28:57.670 * RDB memory usage when created 0.99 Mb
1:S 23 May 2024 10:28:57.670 * Done loading RDB, keys loaded: 0, keys expired: 0.
1:S 23 May 2024 10:28:57.670 * MASTER <-> REPLICA sync: Finished with success
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# 
[root@172-16-120-6 redis-replication]# kubectl  -n redis1 logs redis-replication-2
Redis is running without password which is not recommended
Setting up redis in standalone mode
Running without persistence mode
Running without TLS mode
ACL_MODE is not true, skipping ACL file modification
Starting redis service in standalone mode.....
1:C 23 May 2024 10:27:55.863 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 23 May 2024 10:27:55.863 * Redis version=7.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 23 May 2024 10:27:55.863 * Configuration loaded
1:M 23 May 2024 10:27:55.863 * monotonic clock: POSIX clock_gettime
1:M 23 May 2024 10:27:55.864 # Failed to write PID file: Permission denied
1:M 23 May 2024 10:27:55.864 * Running mode=standalone, port=6379.
1:M 23 May 2024 10:27:55.864 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 23 May 2024 10:27:55.865 * Server initialized
1:M 23 May 2024 10:27:55.865 * Ready to accept connections tcp
1:S 23 May 2024 10:28:51.740 * Before turning into a replica, using my own master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
1:S 23 May 2024 10:28:51.740 * Connecting to MASTER 100.86.3.100:6379
1:S 23 May 2024 10:28:51.740 * MASTER <-> REPLICA sync started
1:S 23 May 2024 10:28:51.740 * REPLICAOF 100.86.3.100:6379 enabled (user request from 'id=17 addr=100.119.38.215:33646 laddr=100.119.38.217:6379 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=46 qbuf-free=20428 argv-mem=23 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=37807 events=r cmd=slaveof user=default redir=-1 resp=3 lib-name=go-redis(,go1.21.10) lib-ver=9.5.1')
1:S 23 May 2024 10:28:51.740 * Non blocking connect for SYNC fired the event.
1:S 23 May 2024 10:28:51.741 * Master replied to PING, replication can continue...
1:S 23 May 2024 10:28:51.741 * Trying a partial resynchronization (request 8ca1fdc42523e7198e3b846d77d9770f2361c0cb:1).
1:S 23 May 2024 10:28:56.719 * Full resync from master: d61d13fbf77bc324bc452135f19d025d3f93354c:14
1:S 23 May 2024 10:28:56.720 * MASTER <-> REPLICA sync: receiving streamed RDB from master with EOF to disk
1:S 23 May 2024 10:28:56.721 * Discarding previously cached master state.
1:S 23 May 2024 10:28:56.721 * MASTER <-> REPLICA sync: Flushing old data
1:S 23 May 2024 10:28:56.721 * MASTER <-> REPLICA sync: Loading DB in memory
1:S 23 May 2024 10:28:56.726 * Loading RDB produced by version 7.2.3
1:S 23 May 2024 10:28:56.726 * RDB age 17 seconds
1:S 23 May 2024 10:28:56.726 * RDB memory usage when created 0.99 Mb
1:S 23 May 2024 10:28:56.726 * Done loading RDB, keys loaded: 0, keys expired: 0.
1:S 23 May 2024 10:28:56.726 * MASTER <-> REPLICA sync: Finished with success
dxygit1 commented 1 month ago

redis-0

redis_version:7.2.3
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:c9995ccd1c9928ca
redis_mode:standalone
os:Linux 3.10.0-1160.el7.x86_64 x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.3.1
process_id:1
process_supervised:no
run_id:6888f65efdf27ad00ef5103e94cc1c87ac60384f
tcp_port:6379
server_time_usec:1716469247388897
uptime_in_seconds:9193
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:5193215
executable:/data/redis-server
config_file:/etc/redis/redis.conf
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,bind=::,port=6379

# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:20480
client_recent_max_output_buffer:20504
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0

# Memory
used_memory:1120856
used_memory_human:1.07M
used_memory_rss:4927488
used_memory_rss_human:4.70M
used_memory_peak:1194928
used_memory_peak_human:1.14M
used_memory_peak_perc:93.80%
used_memory_overhead:886452
used_memory_startup:865760
used_memory_dataset:234404
used_memory_dataset_perc:91.89%
allocator_allocated:1190960
allocator_active:1454080
allocator_resident:5038080
total_system_memory:16655925248
total_system_memory_human:15.51G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.22
allocator_frag_bytes:263120
allocator_rss_ratio:3.46
allocator_rss_bytes:3584000
rss_overhead_ratio:0.98
rss_overhead_bytes:-110592
mem_fragmentation_ratio:4.56
mem_fragmentation_bytes:3847272
mem_not_counted_for_evict:0
mem_replication_backlog:20508
mem_total_replication_buffers:20504
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1716460054
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:331776
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:3650
total_commands_processed:27274
instantaneous_ops_per_sec:2
total_net_input_bytes:1021751
total_net_output_bytes:1224905
total_net_repl_input_bytes:0
total_net_repl_output_bytes:25996
instantaneous_input_kbps:0.09
instantaneous_output_kbps:0.02
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.02
rejected_connections:0
sync_full:2
sync_partial_ok:0
sync_partial_err:2
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:168
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:378
total_forks:1
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:29114
total_writes_processed:9091
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:4
reply_buffer_expands:0
eventloop_cycles:123337
eventloop_duration_sum:12646463
eventloop_duration_cmd_sum:105469
instantaneous_eventloop_cycles_per_sec:10
instantaneous_eventloop_duration_usec:126
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0

# Replication
role:master
connected_slaves:2
slave0:ip=100.89.15.96,port=6379,state=online,offset=12754,lag=1
slave1:ip=100.119.38.217,port=6379,state=online,offset=12754,lag=1
master_failover_state:no-failover
master_replid:d61d13fbf77bc324bc452135f19d025d3f93354c
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:12754
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:12754

# CPU
used_cpu_sys:5.868723
used_cpu_user:9.671467
used_cpu_sys_children:0.003141
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:5.867300
used_cpu_user_main_thread:9.669123

# Modules

# Errorstats

# Cluster
cluster_enabled:0

redis-1

# Server
redis_version:7.2.3
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:c9995ccd1c9928ca
redis_mode:standalone
os:Linux 3.10.0-1160.el7.x86_64 x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.3.1
process_id:1
process_supervised:no
run_id:7c1ba26424f9da8203b14bb1b9f7c888a5c6f841
tcp_port:6379
server_time_usec:1716469309314191
uptime_in_seconds:9235
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:5193277
executable:/data/redis-server
config_file:/etc/redis/redis.conf
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,bind=::,port=6379

# Clients
connected_clients:2
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:24
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0

# Memory
used_memory:1052944
used_memory_human:1.00M
used_memory_rss:4907008
used_memory_rss_human:4.68M
used_memory_peak:1129688
used_memory_peak_human:1.08M
used_memory_peak_perc:93.21%
used_memory_overhead:888492
used_memory_startup:865856
used_memory_dataset:164452
used_memory_dataset_perc:87.90%
allocator_allocated:1156312
allocator_active:1470464
allocator_resident:5189632
total_system_memory:16655925248
total_system_memory_human:15.51G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.27
allocator_frag_bytes:314152
allocator_rss_ratio:3.53
allocator_rss_bytes:3719168
rss_overhead_ratio:0.95
rss_overhead_bytes:-282624
mem_fragmentation_ratio:4.85
mem_fragmentation_bytes:3894720
mem_not_counted_for_evict:0
mem_replication_backlog:20508
mem_total_replication_buffers:20504
mem_clients_slaves:0
mem_clients_normal:1944
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1716460074
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:2758
total_commands_processed:6401
instantaneous_ops_per_sec:0
total_net_input_bytes:200010
total_net_output_bytes:1044607
total_net_repl_input_bytes:13054
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.02
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:1
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:8249
total_writes_processed:13717
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:4
reply_buffer_expands:0
eventloop_cycles:102446
eventloop_duration_sum:17691153
eventloop_duration_cmd_sum:42676
instantaneous_eventloop_cycles_per_sec:9
instantaneous_eventloop_duration_usec:176
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0

# Replication
role:slave
master_host:100.86.3.100
master_port:6379
master_link_status:up
master_last_io_seconds_ago:4
master_sync_in_progress:0
slave_read_repl_offset:12810
slave_repl_offset:12810
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:d61d13fbf77bc324bc452135f19d025d3f93354c
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:12810
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:15
repl_backlog_histlen:12796

# CPU
used_cpu_sys:7.370982
used_cpu_user:13.634999
used_cpu_sys_children:0.000900
used_cpu_user_children:0.000680
used_cpu_sys_main_thread:7.335449
used_cpu_user_main_thread:13.575401

# Modules

# Errorstats

# Cluster
cluster_enabled:0

redis-2

# Server
redis_version:7.2.3
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:c9995ccd1c9928ca
redis_mode:standalone
os:Linux 3.10.0-1160.el7.x86_64 x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.3.1
process_id:1
process_supervised:no
run_id:8da8d60d3016dbfa7a60bab1433b39f73f9f1b46
tcp_port:6379
server_time_usec:1716469339319135
uptime_in_seconds:9264
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:5193307
executable:/data/redis-server
config_file:/etc/redis/redis.conf
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,bind=::,port=6379

# Clients
connected_clients:2
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:24
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0

# Memory
used_memory:1052880
used_memory_human:1.00M
used_memory_rss:4911104
used_memory_rss_human:4.68M
used_memory_peak:1129624
used_memory_peak_human:1.08M
used_memory_peak_perc:93.21%
used_memory_overhead:888428
used_memory_startup:865792
used_memory_dataset:164452
used_memory_dataset_perc:87.90%
allocator_allocated:1156248
allocator_active:1470464
allocator_resident:5189632
total_system_memory:16655917056
total_system_memory_human:15.51G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.27
allocator_frag_bytes:314216
allocator_rss_ratio:3.53
allocator_rss_bytes:3719168
rss_overhead_ratio:0.95
rss_overhead_bytes:-278528
mem_fragmentation_ratio:4.85
mem_fragmentation_bytes:3898880
mem_not_counted_for_evict:0
mem_replication_backlog:20508
mem_total_replication_buffers:20504
mem_clients_slaves:0
mem_clients_normal:1944
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1716460075
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:2767
total_commands_processed:6422
instantaneous_ops_per_sec:0
total_net_input_bytes:200667
total_net_output_bytes:1048162
total_net_repl_input_bytes:13096
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.02
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:1
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:8276
total_writes_processed:13764
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:1
reply_buffer_expands:0
eventloop_cycles:102638
eventloop_duration_sum:16918962
eventloop_duration_cmd_sum:39394
instantaneous_eventloop_cycles_per_sec:9
instantaneous_eventloop_duration_usec:166
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0

# Replication
role:slave
master_host:100.86.3.100
master_port:6379
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_read_repl_offset:12852
slave_repl_offset:12852
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:d61d13fbf77bc324bc452135f19d025d3f93354c
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:12852
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:15
repl_backlog_histlen:12838

# CPU
used_cpu_sys:7.625223
used_cpu_user:12.644562
used_cpu_sys_children:0.000000
used_cpu_user_children:0.001488
used_cpu_sys_main_thread:7.580861
used_cpu_user_main_thread:12.600084

# Modules

# Errorstats

# Cluster
cluster_enabled:0
drivebyer commented 1 month ago

Sentinel and Redis must be in the same namespace.

dxygit1 commented 1 month ago

In the same namespace

image image They are in the same chart, so they must be in the same namespace

drivebyer commented 1 month ago

execute kubectl -n redis1 get redissentinel redis-sentinel -oyaml

dxygit1 commented 1 month ago 
apiVersion: redis.redis.opstreelabs.in/v1beta2
kind: RedisSentinel
metadata:
  annotations:
    meta.helm.sh/release-name: redis-se
    meta.helm.sh/release-namespace: redis1
  creationTimestamp: "2024-05-23T10:27:39Z"
  generation: 1
  labels:
    app.kubernetes.io/component: middleware
    app.kubernetes.io/instance: redis-sentinel
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: redis-sentinel
    app.kubernetes.io/version: 0.16.0
    helm.sh/chart: redis-sentinel-0.16.0
  name: redis-sentinel
  namespace: redis1
  resourceVersion: "97808160"
  uid: 0fdee68f-33cf-42b3-b6d3-02ffbd90b032
spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app.kubernetes.io/name
            operator: In
            values:
            - redis-sentinel
        topologyKey: kubernetes.io/hostname
  clusterSize: 3
  kubernetesConfig:
    image: quay.io/opstree/redis-sentinel:v7.2.3
    imagePullPolicy: IfNotPresent
  livenessProbe:
    failureThreshold: 3
    initialDelaySeconds: 1
    periodSeconds: 10
    successThreshold: 1
    timeoutSeconds: 1
  podSecurityContext:
    fsGroup: 1000
    runAsUser: 1000
  readinessProbe:
    failureThreshold: 3
    initialDelaySeconds: 1
    periodSeconds: 10
    successThreshold: 1
    timeoutSeconds: 1
  redisExporter:
    enabled: false
    image: quay.io/opstree/redis-exporter:v1.44.0
    imagePullPolicy: IfNotPresent
  redisSentinelConfig:
    downAfterMilliseconds: "30000"
    failoverTimeout: "180000"
    masterGroupName: myMaster
    parallelSyncs: "1"
    quorum: "2"
    redisPort: "6379"
    redisReplicationName: redis-sentinel
drivebyer commented 1 month ago

spec.redisSentinelConfig.redisReplicationName must be set to the Redis replication name that needs to be monitored in the same namespace.

dxygit1 commented 1 month ago

Sorry, it was my oversight when making changes. This is indeed the issue. Thank you for your response.