search

RedisLabs / redis-cluster-proxy

A proxy for Redis clusters.
GNU Affero General Public License v3.0
993 stars 132 forks source link

Crash in onClusterNodeDisconnection #56

Open doyoubi opened 4 years ago

doyoubi commented 4 years ago

The version is the release 1.0-beta2.

I will update once I find the way to reproduce it. Hope the traceback can help.

Crash log:

=== PROXY BUG REPORT START: Cut & paste starting from here ===
[2020-04-27 14:24:46.985/0] Redis Cluster Proxy 0.9.102 crashed by signal: 11
[2020-04-27 14:24:46.985/0] Crashed running the instruction at: 0x415756
[2020-04-27 14:24:46.985/0] Accessing address: (nil)
[2020-04-27 14:24:46.985/0] Handling crash on thread: 0

------ STACK TRACE ------
EIP:
./redis-cluster-proxy(onClusterNodeDisconnection+0x16)[0x415756]

Backtrace:
./redis-cluster-proxy(logStackTrace+0x2d)[0x40d1dd]
./redis-cluster-proxy(sigsegvHandler+0x186)[0x40d7f6]
/lib64/libpthread.so.0(+0xf5d0)[0x7f0fb9bc15d0]
./redis-cluster-proxy(onClusterNodeDisconnection+0x16)[0x415756]
./redis-cluster-proxy[0x40a5ab]
./redis-cluster-proxy[0x40a6a1]
./redis-cluster-proxy(resetCluster+0x3e)[0x40a7ce]
./redis-cluster-proxy(updateCluster+0x1d9)[0x40c4e9]
./redis-cluster-proxy[0x41a1f4]
./redis-cluster-proxy(aeProcessEvents+0x291)[0x408f21]
./redis-cluster-proxy(aeMain+0x2b)[0x40920b]
./redis-cluster-proxy[0x41107c]
/lib64/libpthread.so.0(+0x7dd5)[0x7f0fb9bb9dd5]
/lib64/libc.so.6(clone+0x6d)[0x7f0fb98e302d]

------ INFO OUTPUT ------
# Proxy
proxy_version:0.9.102
proxy_git_sha1:00000000
proxy_git_dirty:0
proxy_git_branch:
os:Linux 3.10.0-957.21.3.el7.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:7.3.1
process_id:30681
threads:1
tcp_port:7777
uptime_in_seconds:599
uptime_in_days:0
config_file:proxy.conf
acl_user:default

# Memory
used_memory:2177168
used_memory_human:2.08M
total_system_memory:16479350784
total_system_memory_human:15.35G

# Clients
connected_clients:50
max_clients:10000
thread_0_clinets:50

# Cluster
address:
entry_node::0

---- SIZEOF STRUCTS ----
clientRequest: 184
client: 224
redisClusterConnection: 48
clusterNode: 112
redisCluster: 104
list: 48
listNode: 24
rax: 24
raxNode: 4
raxIterator: 480
aeEventLoop: 88
aeFileEvent: 32
aeTimeEvent: 64

------ REGISTERS ------

RAX:00007f0fb4009340 RBX:00007f0fb40aa3c0
RCX:00007f0fb4008b00 RDX:0000000000000045
RDI:00007f0fb40aa3c0 RSI:00007f0fb97e3b90
RBP:0000000000000000 RSP:00007f0fb97e3b30
R8 :00007f0fb4009340 R9 :00007f0fb4008bc0
R10:00000000fffffc00 R11:00007f0fb9971f40
R12:00007f0fb4008c00 R13:7878787878787878
R14:00000000022d4850 R15:00007f0fb40090a0
RIP:0000000000415756 EFL:0000000000010206
CSGSFS:0000000000000033
(00007f0fb97e3b3f) -> 000000000040a7ce
(00007f0fb97e3b3e) -> 00000000022d4850
(00007f0fb97e3b3d) -> 0000000000000000
(00007f0fb97e3b3c) -> 0000000000000045
(00007f0fb97e3b3b) -> 000000000040a6a1
(00007f0fb97e3b3a) -> 00007f0fb4008c00
(00007f0fb97e3b39) -> 0000000000000000
(00007f0fb97e3b38) -> 00000000022d4850
(00007f0fb97e3b37) -> 000000000040a5ab
(00007f0fb97e3b36) -> 0000000000000001
(00007f0fb97e3b35) -> 00007f0fb4008c00
(00007f0fb97e3b34) -> 0000000000000000
(00007f0fb97e3b33) -> 00007f0fb40aa3c0
(00007f0fb97e3b32) -> 0000000000000001
(00007f0fb97e3b31) -> 00007f0fb4019930
(00007f0fb97e3b30) -> 00007f0fb40090a0

------ DUMPING CODE AROUND EIP ------
Symbol: onClusterNodeDisconnection (base: 0x415740)
Module: ./redis-cluster-proxy (base 0x400000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x415740 -D -b binary -m i386:x86-64 /tmp/dump.bin
------
dump of function  (hexdump of 150 bytes):
4155415455534883ec184c8b2f4d85ed0f8490010000498b45004885c00f84830100008bb08400000085f60f8875010000488b4708488b151c4e22004889fd486308488b14ca4885d27445488b7a204885ff743cba03000000e8f232ffff488b450841c7451c000000004885c07529bfe2ba4200ba330c0000be60b74200e8cd82ffffbf01000000e89312ffff0f1f0041c7451c0000
Function at 0x408a90 is aeDeleteFileEvent
Function at 0x40da90 is _proxyAssert

=== PROXY BUG REPORT END. Make sure to include from START to END. ===