[3.] One redis master becomes not reachable (link cut, TCP retransmissions)
[4.] User code of redis-plus-plus detects that for 4 seconds there is no response for those requests that are directed to the unreachable redis (based on hash slot)
[5.] User code of redis-plus-plus initiates AsyncRedisCluster reset with ip-address / port of a reachable redis master (Note: if the client does not make AsyncRedisCluster reset then the traffic towards the unreachable redis master will fail for minutes)
[6.] traffic stabilised, it seems everything is Ok
[7.] At the end of the test scenario valgrind reports memory leak:
!! Valgrind error.
==1553== Memcheck, a memory error detector
==1553== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==1553== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==1553== Command: /sources/**
==1553== Parent PID: 1552
==1553==
==1553== Warning: invalid file descriptor -1 in syscall close()
==1553==
==1553== HEAP SUMMARY:
==1553== in use at exit: 245,588 bytes in 2,720 blocks
==1553== total heap usage: 6,474,015 allocs, 6,471,295 frees, 1,165,728,660 bytes allocated
==1553==
==1553== 4,022 (24 direct, 3,998 indirect) bytes in 1 blocks are definitely lost in loss record 105 of 122
==1553== at 0x4C388C3: operator new(unsigned long) (vg_replace_malloc.c:422)
==1553== by 0x7C1D861: sw::redis::AsyncConnection::_connect(sw::redis::ConnectionOptions const&) (async_connection.cpp:579)
==1553== by 0x7C1EC46: sw::redis::AsyncConnection::_connect() (async_connection.cpp:477)
==1553== by 0x7C275D8: sw::redis::EventLoop::_event_callback(uv_async_s) (event_loop.cpp:146)
==1553== by 0x7E6F2F0: ??? (in /usr/lib64/libuv.so.1.0.0)
==1553== by 0x7E80D14: uvio_poll (in /usr/lib64/libuv.so.1.0.0)
==1553== by 0x7E6FA73: uv_run (in /usr/lib64/libuv.so.1.0.0)
==1553== by 0x8D55B22: ??? (in /usr/lib64/libstdc++.so.6.0.25)
==1553== by 0x83441C9: start_thread (in /usr/lib64/libpthread-2.28.so)
==1553== by 0x95FBE72: clone (in /usr/lib64/libc-2.28.so)
==1553==
==1553== 8,046 (464 direct, 7,582 indirect) bytes in 1 blocks are definitely lost in loss record 111 of 122
==1553== at 0x4C3D096: realloc (vg_replace_malloc.c:1437)
==1553== by 0x778877A: redisAsyncConnectWithOptions (in /usr/lib64/libhiredis.so.1.1.0)
==1553== by 0x7C1D80D: sw::redis::AsyncConnection::_connect(sw::redis::ConnectionOptions const&) (async_connection.cpp:569)
==1553== by 0x7C1EC46: sw::redis::AsyncConnection::_connect() (async_connection.cpp:477)
==1553== by 0x7C275D8: sw::redis::EventLoop::_event_callback(uv_async_s*) (event_loop.cpp:146)
==1553== by 0x7E6F2F0: ??? (in /usr/lib64/libuv.so.1.0.0)
==1553== by 0x7E80D14: uvio_poll (in /usr/lib64/libuv.so.1.0.0)
==1553== by 0x7E6FA73: uv_run (in /usr/lib64/libuv.so.1.0.0)
==1553== by 0x8D55B22: ??? (in /usr/lib64/libstdc++.so.6.0.25)
==1553== by 0x83441C9: start_thread (in /usr/lib64/libpthread-2.28.so)
==1553== by 0x95FBE72: clone (in /usr/lib64/libc-2.28.so)
==1553==
==1553== LEAK SUMMARY:
==1553== definitely lost: 488 bytes in 2 blocks
==1553== indirectly lost: 11,580 bytes in 70 blocks
==1553== possibly lost: 197,667 bytes in 2,360 blocks
==1553== still reachable: 35,853 bytes in 288 blocks
==1553== of which reachable via heuristic:
==1553== multipleinheritance: 18,096 bytes in 377 blocks
==1553== suppressed: 0 bytes in 0 blocks
==1553== Reachable blocks (those to which a pointer was found) are not shown.
==1553== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==1553==
==1553== For lists of detected and suppressed errors, rerun with: -s
==1553== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Expected behavior
No memory leak after reset operation.
Environment:
OS: Rocky Linux 8.2-20.el8.0.1
Compiler: gcc version 8.5.0
hiredis version: hiredis 1.2.0
redis-plus-plus version: 1.3.12
Additional context
Redis cluster is used with 3 masters and 3 slaves.
Describe the bug AsyncRedisCluster reset causes memory leak if one of the redis master becomes unreachable.
To Reproduce [1.] asynch client is defined / used in the following way:
::std::shared_ptr<::sw::redis::AsyncRedisCluster> m_redis_cluster; m_redis_cluster.reset(new ::sw::redis::AsyncRedisCluster(opts, pool_opts, ::sw::redis::Role::MASTER));
[2.] Continuous traffic is generated
[3.] One redis master becomes not reachable (link cut, TCP retransmissions)
[4.] User code of redis-plus-plus detects that for 4 seconds there is no response for those requests that are directed to the unreachable redis (based on hash slot)
[5.] User code of redis-plus-plus initiates AsyncRedisCluster reset with ip-address / port of a reachable redis master (Note: if the client does not make AsyncRedisCluster reset then the traffic towards the unreachable redis master will fail for minutes)
m_redis_cluster.reset(new ::sw::redis::AsyncRedisCluster(opts, pool_opts, ::sw::redis::Role::MASTER));
[6.] traffic stabilised, it seems everything is Ok
[7.] At the end of the test scenario valgrind reports memory leak:
!! Valgrind error. ==1553== Memcheck, a memory error detector ==1553== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==1553== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info ==1553== Command: /sources/** ==1553== Parent PID: 1552 ==1553== ==1553== Warning: invalid file descriptor -1 in syscall close() ==1553== ==1553== HEAP SUMMARY: ==1553== in use at exit: 245,588 bytes in 2,720 blocks ==1553== total heap usage: 6,474,015 allocs, 6,471,295 frees, 1,165,728,660 bytes allocated ==1553== ==1553== 4,022 (24 direct, 3,998 indirect) bytes in 1 blocks are definitely lost in loss record 105 of 122 ==1553== at 0x4C388C3: operator new(unsigned long) (vg_replace_malloc.c:422) ==1553== by 0x7C1D861: sw::redis::AsyncConnection::_connect(sw::redis::ConnectionOptions const&) (async_connection.cpp:579) ==1553== by 0x7C1EC46: sw::redis::AsyncConnection::_connect() (async_connection.cpp:477) ==1553== by 0x7C275D8: sw::redis::EventLoop::_event_callback(uv_async_s) (event_loop.cpp:146) ==1553== by 0x7E6F2F0: ??? (in /usr/lib64/libuv.so.1.0.0) ==1553== by 0x7E80D14: uvio_poll (in /usr/lib64/libuv.so.1.0.0) ==1553== by 0x7E6FA73: uv_run (in /usr/lib64/libuv.so.1.0.0) ==1553== by 0x8D55B22: ??? (in /usr/lib64/libstdc++.so.6.0.25) ==1553== by 0x83441C9: start_thread (in /usr/lib64/libpthread-2.28.so) ==1553== by 0x95FBE72: clone (in /usr/lib64/libc-2.28.so) ==1553== ==1553== 8,046 (464 direct, 7,582 indirect) bytes in 1 blocks are definitely lost in loss record 111 of 122 ==1553== at 0x4C3D096: realloc (vg_replace_malloc.c:1437) ==1553== by 0x778877A: redisAsyncConnectWithOptions (in /usr/lib64/libhiredis.so.1.1.0) ==1553== by 0x7C1D80D: sw::redis::AsyncConnection::_connect(sw::redis::ConnectionOptions const&) (async_connection.cpp:569) ==1553== by 0x7C1EC46: sw::redis::AsyncConnection::_connect() (async_connection.cpp:477) ==1553== by 0x7C275D8: sw::redis::EventLoop::_event_callback(uv_async_s*) (event_loop.cpp:146) ==1553== by 0x7E6F2F0: ??? (in /usr/lib64/libuv.so.1.0.0) ==1553== by 0x7E80D14: uvio_poll (in /usr/lib64/libuv.so.1.0.0) ==1553== by 0x7E6FA73: uv_run (in /usr/lib64/libuv.so.1.0.0) ==1553== by 0x8D55B22: ??? (in /usr/lib64/libstdc++.so.6.0.25) ==1553== by 0x83441C9: start_thread (in /usr/lib64/libpthread-2.28.so) ==1553== by 0x95FBE72: clone (in /usr/lib64/libc-2.28.so) ==1553== ==1553== LEAK SUMMARY: ==1553== definitely lost: 488 bytes in 2 blocks ==1553== indirectly lost: 11,580 bytes in 70 blocks ==1553== possibly lost: 197,667 bytes in 2,360 blocks ==1553== still reachable: 35,853 bytes in 288 blocks ==1553== of which reachable via heuristic: ==1553== multipleinheritance: 18,096 bytes in 377 blocks ==1553== suppressed: 0 bytes in 0 blocks ==1553== Reachable blocks (those to which a pointer was found) are not shown. ==1553== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==1553== ==1553== For lists of detected and suppressed errors, rerun with: -s ==1553== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Expected behavior No memory leak after reset operation.
Environment: OS: Rocky Linux 8.2-20.el8.0.1 Compiler: gcc version 8.5.0 hiredis version: hiredis 1.2.0 redis-plus-plus version: 1.3.12
Additional context Redis cluster is used with 3 masters and 3 slaves.