search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
9.01k stars 1.08k forks source link

[YSQL] fix memory leak in ysql_dump #24934

Open mdbridge opened 2 days ago

mdbridge commented 2 days ago

Jira Link: DB-14076

Description

The following test:

ybd asan --cxx-test integration-tests_xcluster_ddl_replication-test --gtest_filter XClusterDDLReplicationTest.Bootstrapping

finds a memory leak in ysql_dump.

This task is to find and fix that leak.

I have muted that test for the ASAN build for now.

Details of the leak report:

[P-yb-controller-1] ==23626==ERROR: LeakSanitizer: detected memory leaks
[P-yb-controller-1] 
[P-yb-controller-1] Indirect leak of 6144 byte(s) in 3 object(s) allocated from:
[P-yb-controller-1]     #0 0x5575dc8caaee in malloc /opt/yb-build/llvm/yb-llvm-v17.0.6-yb-1-1726891784-9b881774-ubuntu24.04-x86_64-build/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
[P-yb-controller-1]     #1 0x7f05a9970c2c in pqResultAlloc /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:633:28
[P-yb-controller-1]     #2 0x7f05a998bcdf in getRowDescriptions /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-protocol3.c:544:4
[P-yb-controller-1]     #3 0x7f05a9989c0b in pqParseInput3 /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-protocol3.c:324:11
[P-yb-controller-1]     #4 0x7f05a9974908 in parseInput /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2014:2
[P-yb-controller-1]     #5 0x7f05a9974908 in PQgetResult /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2100:3
[P-yb-controller-1]     #6 0x7f05a99759c7 in PQexecFinish /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2417:19
[P-yb-controller-1]     #7 0x7f05a99759c7 in PQexec /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2256:9
[P-yb-controller-1]     #8 0x5575dc99bc2f in ExecuteSqlQuery /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_backup_db.c:296:8
[P-yb-controller-1]     #9 0x5575dc99b856 in ExecuteSqlQueryForSingleRow /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_backup_db.c:311:8
[P-yb-controller-1]     #10 0x5575dc95d980 in binary_upgrade_set_pg_class_oids /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:4987:16
[P-yb-controller-1]     #11 0x5575dc95ec88 in dumpTableSchema /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:15676:5
[P-yb-controller-1]     #12 0x5575dc9198e0 in dumpTable /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:15223:4
[P-yb-controller-1]     #13 0x5575dc9198e0 in dumpDumpableObject /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:10145:4
[P-yb-controller-1]     #14 0x5575dc9198e0 in main /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:1007:3
[P-yb-controller-1]     #15 0x7f05a91a81c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
[P-yb-controller-1]     #16 0x7f05a91a828a in __libc_start_main csu/../csu/libc-start.c:360:3
[P-yb-controller-1]     #17 0x5575dc82d474 in _start (/home/mdbridge/code/yugabyte-db/build/asan-clang17-dynamic-ninja/postgres/bin/ysql_dump+0x10c474)
[P-yb-controller-1] 
[P-yb-controller-1] Objects leaked above:
[P-yb-controller-1] 0x51d0001f7c80 (2048 bytes)
[P-yb-controller-1] 0x51d0001fa480 (2048 bytes)
[P-yb-controller-1] 0x51d0001fd680 (2048 bytes)
[P-yb-controller-1] 
[P-yb-controller-1] Indirect leak of 3072 byte(s) in 3 object(s) allocated from:
[P-yb-controller-1]     #0 0x5575dc8caaee in malloc /opt/yb-build/llvm/yb-llvm-v17.0.6-yb-1-1726891784-9b881774-ubuntu24.04-x86_64-build/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
[P-yb-controller-1]     #1 0x7f05a9971064 in pqAddTuple /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:1044:5
[P-yb-controller-1]     #2 0x7f05a99728a7 in pqRowProcessor /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:1277:7
[P-yb-controller-1]     #3 0x7f05a998cd68 in getAnotherTuple /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-protocol3.c:832:6
[P-yb-controller-1]     #4 0x7f05a99895d6 in pqParseInput3 /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-protocol3.c:378:11
[P-yb-controller-1]     #5 0x7f05a9974908 in parseInput /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2014:2
[P-yb-controller-1]     #6 0x7f05a9974908 in PQgetResult /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2100:3
[P-yb-controller-1]     #7 0x7f05a99759c7 in PQexecFinish /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2417:19
[P-yb-controller-1]     #8 0x7f05a99759c7 in PQexec /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2256:9
[P-yb-controller-1]     #9 0x5575dc99bc2f in ExecuteSqlQuery /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_backup_db.c:296:8
[P-yb-controller-1]     #10 0x5575dc99b856 in ExecuteSqlQueryForSingleRow /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_backup_db.c:311:8
[P-yb-controller-1]     #11 0x5575dc95d980 in binary_upgrade_set_pg_class_oids /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:4987:16
[P-yb-controller-1]     #12 0x5575dc95ec88 in dumpTableSchema /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:15676:5
[P-yb-controller-1]     #13 0x5575dc9198e0 in dumpTable /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:15223:4
[P-yb-controller-1]     #14 0x5575dc9198e0 in dumpDumpableObject /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:10145:4
[P-yb-controller-1]     #15 0x5575dc9198e0 in main /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:1007:3
[P-yb-controller-1]     #16 0x7f05a91a81c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
[P-yb-controller-1]     #17 0x7f05a91a828a in __libc_start_main csu/../csu/libc-start.c:360:3
[P-yb-controller-1]     #18 0x5575dc82d474 in _start (/home/mdbridge/code/yugabyte-db/build/asan-clang17-dynamic-ninja/postgres/bin/ysql_dump+0x10c474)
[P-yb-controller-1] 
[P-yb-controller-1] Objects leaked above:
[P-yb-controller-1] 0x519000012780 (1024 bytes)
[P-yb-controller-1] 0x519000014580 (1024 bytes)
[P-yb-controller-1] 0x519000015e80 (1024 bytes)
[P-yb-controller-1] 
[P-yb-controller-1] Indirect leak of 648 byte(s) in 3 object(s) allocated from:
[P-yb-controller-1]     #0 0x5575dc8caaee in malloc /opt/yb-build/llvm/yb-llvm-v17.0.6-yb-1-1726891784-9b881774-ubuntu24.04-x86_64-build/src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
[P-yb-controller-1]     #1 0x7f05a996dd70 in PQmakeEmptyPGresult /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:164:24
[P-yb-controller-1]     #2 0x7f05a998bb2d in getRowDescriptions /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-protocol3.c
[P-yb-controller-1]     #3 0x7f05a9989c0b in pqParseInput3 /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-protocol3.c:324:11
[P-yb-controller-1]     #4 0x7f05a9974908 in parseInput /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2014:2
[P-yb-controller-1]     #5 0x7f05a9974908 in PQgetResult /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2100:3
[P-yb-controller-1]     #6 0x7f05a99759c7 in PQexecFinish /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2417:19
[P-yb-controller-1]     #7 0x7f05a99759c7 in PQexec /home/mdbridge/code/yugabyte-db/src/postgres/src/interfaces/libpq/../../../../../../src/postgres/src/interfaces/libpq/fe-exec.c:2256:9
[P-yb-controller-1]     #8 0x5575dc99bc2f in ExecuteSqlQuery /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_backup_db.c:296:8
[P-yb-controller-1]     #9 0x5575dc99b856 in ExecuteSqlQueryForSingleRow /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_backup_db.c:311:8
[P-yb-controller-1]     #10 0x5575dc95d980 in binary_upgrade_set_pg_class_oids /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:4987:16
[P-yb-controller-1]     #11 0x5575dc95ec88 in dumpTableSchema /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:15676:5
[P-yb-controller-1]     #12 0x5575dc9198e0 in dumpTable /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:15223:4
[P-yb-controller-1]     #13 0x5575dc9198e0 in dumpDumpableObject /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:10145:4
[P-yb-controller-1]     #14 0x5575dc9198e0 in main /home/mdbridge/code/yugabyte-db/src/postgres/src/bin/pg_dump/../../../../../../src/postgres/src/bin/pg_dump/pg_dump.c:1007:3
[P-yb-controller-1]     #15 0x7f05a91a81c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
[P-yb-controller-1]     #16 0x7f05a91a828a in __libc_start_main csu/../csu/libc-start.c:360:3
[P-yb-controller-1]     #17 0x5575dc82d474 in _start (/home/mdbridge/code/yugabyte-db/build/asan-clang17-dynamic-ninja/postgres/bin/ysql_dump+0x10c474)
[P-yb-controller-1] 
[P-yb-controller-1] Objects leaked above:
[P-yb-controller-1] 0x51100007aac0 (216 bytes)
[P-yb-controller-1] 0x51100007bb00 (216 bytes)
[P-yb-controller-1] 0x51100008fb00 (216 bytes)
[P-yb-controller-1] 
[P-yb-controller-1] -----------------------------------------------------
[P-yb-controller-1] Suppressions used:
[P-yb-controller-1]   count      bytes template
[P-yb-controller-1]      16        157 pg_malloc_internal
[P-yb-controller-1] -----------------------------------------------------
[P-yb-controller-1] 
[P-yb-controller-1] SUMMARY: AddressSanitizer: 9864 byte(s) leaked in 9 allocation(s).

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

mdbridge commented 11 hours ago

this leak also causes the following tests to fail: XClusterAutomaticModeTest.SequenceReplicationBootstrappingBumpInMiddle XClusterAutomaticModeTest.SequenceReplicationBootstrappingWith2Databases XClusterAutomaticModeTest.SequenceReplicationBootstrappingWithoutBumps

muted them on asan as well