Memory leaks detected in psqlodbc library

[progmachine]

Hello, I am working on my C++ applications database connectivity using ODBC. Recently i began running my unit tests with address sanitizer enabled, and LeakSanitizer (used by default with ASAN) detected two points of memory leaks within psqlodbc.

1. The first leak is simple, detected with password field, but i suspect it relates to 3 points of memory leaks with the same principle (ci->password, ci->conn_settings, ci->pqopt): in file dlg_specific.c, function copyConnAttributes, lines 627, 673, 678. If these parameters have been set in data source config, and also provided with connection string, then previous values will be just lost, causing memory leak. It is better to use NULL_THE_NAME macro before assigning new values. I placed macro in correct places, and it fixed this first memory leak point.

2. The second leak is much more complex, leaked objects allocated at parse.c:908, when using SQLDescribeCol function after 'select' SQL-statement, SQLDescribeCol will call getColumnsInfo down the execution code path. This function will allocate COL_INFO objects and place them in ConnectionClass object as columns information cache. Next step to reproduce this bug, is to call 'drop table' SQL-statement in the same connection, it will detect that cache is obsolete and call CC_clear_col_info function on ConnectionClass object, to clear the colinfo cache, but because it is not final destroy, it will see to refcnt field of COL_INFO objects, and leave this objects not destroyed, just detaching them from ConnectionClass object.

The second leak is complex, because leaked objects use reference counting lifetime management, and it is not obvious to me where to look for another points in code, where this lifetime management should be done. May be it's simple refcnt decrement was forgotten in CC_clear_col_info, may be not. For example, StatementClass objects also have references to these COL_INFO objects, but it seems not doing any refcnt lifetime management of COL_INFO objects. If we simply add decrement in CC_clear_col_info function, it will leave zombie references in StatementClass objects. I suspect that this refcnt lifetime management of COL_INFO objects is completely broken...

[davecramer]

Can you provide a PR for these fixes ?

[progmachine]

I can easily provide PR to fix the first memory leak point, in a few days, may be Sunday or Monday. But i can't provide fix for the second leak point, because i don't know how to fix it correctly. If no one can fix it, i can do this job, but it will not be fast, and potentially can introduce bugs :( Because it's new codebase for me, I don't know it's design, and it's pure C code, without classes, constructors/destructors etc... What we need to do in this situation?

[davecramer]

@apgrucza Any chance you can look at this. You have been in the code around allocating memory more than anyone lately ?

[apgrucza]

I feel much the same as @progmachine in that I don't really understand the relationships between the various structs and the intended design around ownership of allocated memory. The mimalloc changes I made were simply adding a drop-in replacement for malloc which did not require me to understand all this. Without modern language features such a destructors and smart pointers, I'm not surprised there are memory leaks.

When I added mimalloc and enabled MIMALLOC_SHOW_STATS, I did notice that it reported memory leaks (and the reports were more detailed in debug builds). But the problem does not lie with detecting memory leaks but knowing the correct way to fix them. I noticed the _MEMORY_DEBUG_ symbol was added by @hiinoue for the purposes of finding memory leaks, but that was 18 years ago so I doubt he'd be able to assist now.

If @progmachine were to attempt a fix, I wonder whether the current tests are comprehensive enough to guarantee that it does not introduce any bugs.

[davecramer]

@apgrucza OK, can you provide the report with all of the memory leaks ? I think we need to try to fix this, and no I don't think @hiinoue is available any more.

[apgrucza]

Leaks were detected in the following tests:

11 - premature.txt 13 - param-conversions.txt 27 - cursor-name.txt 40 - diagnostic.txt 44 - odbc-escapes.txt 47 - fetch-refcursors.txt

As you can see, mimalloc doesn't tell you where in the code the memory leaks are. Although it can integrate with memory tracking tools, the readme recommends using the standard allocator to debug memory leaks. @progmachine are you able to provide a report from your tool whilst running the psqlODBC tests?

[davecramer]

so looking at the code

for (i = 0; i < self->ntables; i++)
        {
            if (coli = self->col_info[i], NULL != coli)
            {
                if (destroy || coli->refcnt == 0)
                {
                    free_col_info_contents(coli);
                    free(coli);
                    self->col_info[i] = NULL;
                }
                else
                    coli->acc_time = 0;
            }
        }
        self->ntables = 0;

The issue is that we set ntables to 0 regardless if the column info has been freed or not.

The naive solution to this is to re-order the column info as we delete them and use the correct value for ntables.

Thoughts ?

[progmachine]

That's incorrect. What this function doing, is completely "releasing" COL_INFO objects from ConnectionClass object. No mater if these objects actually destroyed or not, connection object will not have any one of them. So it's just zeroing self->ntables, and it is correct. The problem is:

• if "destroy" is true (when connection is closing), COL_INFO objects will always be deleted. That's good.
• but if "destroy" is false (col_info cache is obsolete), COL_INFO objects will not be deleted, just detached from ConnectionClass object, without correct releasing using refcnt. That's bad :(

Today i finally had some spare time to begin working on psqlodbc project, in the near future i will post detailed memory leaks report from tests. And maybe will begin fixing these issues.

[davecramer]

That's incorrect. What this function doing, is completely "releasing" COL_INFO objects from ConnectionClass object. No mater if these objects actually destroyed or not, connection object will not have any one of them. So it's just zeroing self->ntables, and it is correct. The problem is:

• if "destroy" is true (when connection is closing), COL_INFO objects will always be deleted. That's good.
• but if "destroy" is false (col_info cache is obsolete), COL_INFO objects will not be deleted, just detached from ConnectionClass object, without correct releasing using refcnt. That's bad :(

Help me out here, where are they "released" ?

Today i finally had some spare time to begin working on psqlodbc project, in the near future i will post detailed memory leaks report from tests. And maybe will begin fixing these issues.

That would be excellent!

[progmachine]

Help me out here, where are they "released" ?

I took word "release" in quotes, with purpuse to indicate, that this action is done with deviations, and must be fixed. But overall meaning is, that COL_INFO objects must be released from ConnectionClass object with refcnt mechanizm, and anyhow ConnectionClass object will not have any of them, so operand self->ntables = 0; is correct.

[davecramer]

I'm trying to see where they are detached? All I see is that if refcount is not zero they are skipped and still self->ntables is set to 0 suggesting that there are no tables in the ConnectionClass object

[progmachine]

... and still self->ntables is set to 0 suggesting that there are no tables in the ConnectionClass object

CC_clear_col_info function makes exactly that: ConnectionClass object will not have any cached COL_INFO objects, after it's called.

All I see is that if refcount is not zero they are skipped

refcnt lifetime management works like that - if refcnt is not zero then somewhere also is reference to object exist, and object must be destroyed somewhere else, where refcnt will be 0. std::shared_ptr in C++ works with the same principle.

I strongly suspect, that this refcnt mechanize is broken, and destroy function parameter used in if statement alongside with refcnt, is a crutch, designed to avoid rude memory leaks, while working by standard way.

[davecramer]

see https://github.com/postgresql-interfaces/psqlodbc/pull/18 for a possible fix. I agree the refcount is likely broken, but first we would need tests to make sure we don't make it workse

[progmachine]

First, partially successful run of tests, produced leaks. Some tests failed with messages AddressSanitizer:DEADLYSIGNAL and did not produced any results. I think, we need to debug all tests set with ASAN enabled, alongside with memleaks fixing :) And i feel, it is a lot of work :-\

[davecramer]

So this shows where they were allocated, but doesn't show where the leak occurs ?

[progmachine]

So this shows where they were allocated, but doesn't show where the leak occurs ?

Yes, it's impossible to automatically detect cause of leak. But detailed info about what allocation leaked, greatly helps to find it's cause, drastically narrowing down search field.

[davecramer]

FYI, running make installcheck on my machine works fine, well almost fine as my postgres is not built with XML

[progmachine]

I know. First unsuccessful run of tests was my fault - incorrect odbc.ini config. If just running tests, they will be successful, but enabling LeakSanitizer by preloading dynamic library, cause fails for many tests. I need to fix it, before continuing.

[progmachine]

Managed to run tests with AddressSanitizer+LeakSanitizer. It appears that on Ubuntu 22.04 with GCC 12 ASAN+LeakSan is incompatible with new kernel ASLR randomization. Reducing randomization entropy helped These are complete leaks report from tests + heap buffer overflow.

connect .................... ok   
stmthandles ................ ok   
select ..................... ok   
update ..................... ok   
commands ................... ok   
multistmt .................. ok   
getresult .................. ok   
colattribute ............... ok   
result-conversions ......... ok   
prepare .................... ok   
premature .................. 
=================================================================
==52495==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1280 byte(s) in 4 object(s) allocated from:
    #0 0x7f75336bf91f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f7532a96990 in QR_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:184
    #2 0x7f7532abe654 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2953
    #3 0x7f7532a7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #4 0x7f7532a7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #5 0x7f7532ab7e46 in SC_describe /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:1160
    #6 0x7f7532a9b9f4 in SC_describe_ok /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:95
    #7 0x7f7532a9bf27 in PGAPI_NumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:188
    #8 0x7f7532ac8d28 in SQLNumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:552
    #9 0x7f7533cee813 in SQLNumResultCols (/lib/x86_64-linux-gnu/libodbc.so.2+0x1d813)

Direct leak of 640 byte(s) in 2 object(s) allocated from:
    #0 0x7f75336bf91f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f7532a96990 in QR_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:184
    #2 0x7f7532abe654 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2953
    #3 0x7f7532a7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #4 0x7f7532a7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #5 0x7f7532ab7e64 in SC_describe /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:1164
    #6 0x7f7532a9b9f4 in SC_describe_ok /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:95
    #7 0x7f7532a9bf27 in PGAPI_NumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:188
    #8 0x7f7532ac8d28 in SQLNumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:552
    #9 0x7f7533cee813 in SQLNumResultCols (/lib/x86_64-linux-gnu/libodbc.so.2+0x1d813)

Indirect leak of 192 byte(s) in 2 object(s) allocated from:
    #0 0x7f75336bf2e7 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f7532a6bb5b in CI_set_num_fields /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:146
    #2 0x7f7532a6b828 in CI_read_fields_from_pgres /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:74
    #3 0x7f7532abecf1 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:3038
    #4 0x7f7532a7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #5 0x7f7532a7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #6 0x7f7532ab7e64 in SC_describe /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:1164
    #7 0x7f7532a9b9f4 in SC_describe_ok /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:95
    #8 0x7f7532a9bf27 in PGAPI_NumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:188
    #9 0x7f7532ac8d28 in SQLNumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:552
    #10 0x7f7533cee813 in SQLNumResultCols (/lib/x86_64-linux-gnu/libodbc.so.2+0x1d813)

Indirect leak of 128 byte(s) in 4 object(s) allocated from:
    #0 0x7f75336bf2e7 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f7532a6bb5b in CI_set_num_fields /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:146
    #2 0x7f7532a6b828 in CI_read_fields_from_pgres /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:74
    #3 0x7f7532abecf1 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:3038
    #4 0x7f7532a7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #5 0x7f7532a7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #6 0x7f7532ab7e46 in SC_describe /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:1160
    #7 0x7f7532a9b9f4 in SC_describe_ok /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:95
    #8 0x7f7532a9bf27 in PGAPI_NumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:188
    #9 0x7f7532ac8d28 in SQLNumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:552
    #10 0x7f7533cee813 in SQLNumResultCols (/lib/x86_64-linux-gnu/libodbc.so.2+0x1d813)

Indirect leak of 64 byte(s) in 4 object(s) allocated from:
    #0 0x7f75336bf91f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f7532a6b715 in CI_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:30
    #2 0x7f7532a969c5 in QR_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:195
    #3 0x7f7532abe654 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2953
    #4 0x7f7532a7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #5 0x7f7532a7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #6 0x7f7532ab7e46 in SC_describe /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:1160
    #7 0x7f7532a9b9f4 in SC_describe_ok /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:95
    #8 0x7f7532a9bf27 in PGAPI_NumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:188
    #9 0x7f7532ac8d28 in SQLNumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:552
    #10 0x7f7533cee813 in SQLNumResultCols (/lib/x86_64-linux-gnu/libodbc.so.2+0x1d813)

Indirect leak of 54 byte(s) in 6 object(s) allocated from:
    #0 0x7f7533673668 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:439
    #1 0x7f7532a6bbcd in CI_set_field_info /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:160
    #2 0x7f7532a6ba2b in CI_read_fields_from_pgres /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:108
    #3 0x7f7532abecf1 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:3038
    #4 0x7f7532a7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #5 0x7f7532a7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #6 0x7f7532ab7e64 in SC_describe /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:1164
    #7 0x7f7532a9b9f4 in SC_describe_ok /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:95
    #8 0x7f7532a9bf27 in PGAPI_NumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:188
    #9 0x7f7532ac8d28 in SQLNumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:552
    #10 0x7f7533cee813 in SQLNumResultCols (/lib/x86_64-linux-gnu/libodbc.so.2+0x1d813)

Indirect leak of 32 byte(s) in 2 object(s) allocated from:
    #0 0x7f75336bf91f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f7532a6b715 in CI_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:30
    #2 0x7f7532a969c5 in QR_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:195
    #3 0x7f7532abe654 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2953
    #4 0x7f7532a7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #5 0x7f7532a7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #6 0x7f7532ab7e64 in SC_describe /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:1164
    #7 0x7f7532a9b9f4 in SC_describe_ok /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:95
    #8 0x7f7532a9bf27 in PGAPI_NumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:188
    #9 0x7f7532ac8d28 in SQLNumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:552
    #10 0x7f7533cee813 in SQLNumResultCols (/lib/x86_64-linux-gnu/libodbc.so.2+0x1d813)

Indirect leak of 32 byte(s) in 4 object(s) allocated from:
    #0 0x7f7533673668 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:439
    #1 0x7f7532a6bbcd in CI_set_field_info /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:160
    #2 0x7f7532a6ba2b in CI_read_fields_from_pgres /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:108
    #3 0x7f7532abecf1 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:3038
    #4 0x7f7532a7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #5 0x7f7532a7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #6 0x7f7532ab7e46 in SC_describe /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:1160
    #7 0x7f7532a9b9f4 in SC_describe_ok /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:95
    #8 0x7f7532a9bf27 in PGAPI_NumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/results.c:188
    #9 0x7f7532ac8d28 in SQLNumResultCols /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:552
    #10 0x7f7533cee813 in SQLNumResultCols (/lib/x86_64-linux-gnu/libodbc.so.2+0x1d813)

SUMMARY: AddressSanitizer: 2422 byte(s) leaked in 28 allocation(s).

params ..................... ok   
param-conversions .......... ok   
parse ...................... ok   
identity ................... ok   
notice ..................... ok   
arraybinding ............... ok   
insertreturning ............ ok   
dataatexecution ............ ok   
boolsaschar ................ ok   
cvtnulldate ................ ok   
alter ...................... ok   
quotes ..................... ok   
cursors .................... ok   
cursor-movement ............ ok   
cursor-commit .............. ok   
cursor-name ................ 
=================================================================
==52599==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 22 byte(s) in 1 object(s) allocated from:
    #0 0x7f8d88873668 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:439
    #1 0x7f8d87c7bdce in copy_statement_with_parameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2992
    #2 0x7f8d87c8c403 in Exec_with_parameters_resolved /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:473
    #3 0x7f8d87c8e612 in PGAPI_Execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:1182
    #4 0x7f8d87c8bb65 in PGAPI_ExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:191
    #5 0x7f8d87cc83d0 in SQLExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:322
    #6 0x7f8d88f52812 in SQLExecDirect (/lib/x86_64-linux-gnu/libodbc.so.2+0x16812)

SUMMARY: AddressSanitizer: 22 byte(s) leaked in 1 allocation(s).

cursor-block-delete ........
=================================================================
==52607==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300002d027 at pc 0x7f5154c4992a bp 0x7ffeeaabc480 sp 0x7ffeeaabbc28
READ of size 23 at 0x60300002d027 thread T0
    #0 0x7f5154c49929 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
    #1 0x7f5153e7c8b7 in copy_statement_with_parameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:3162
    #2 0x7f5153e8c403 in Exec_with_parameters_resolved /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:473
    #3 0x7f5153e8e612 in PGAPI_Execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:1182
    #4 0x7f5153e8bb65 in PGAPI_ExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:191
    #5 0x7f5153ec83d0 in SQLExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:322
    #6 0x7f5154b94812 in SQLExecDirect (/lib/x86_64-linux-gnu/libodbc.so.2+0x16812)
    #7 0x55c49211dafa in main src/cursor-block-delete-test.c:107
    #8 0x7f5154829d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #9 0x7f5154829e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #10 0x55c49211d424 in _start (/home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/test/exe/cursor-block-delete-test+0x1424)

0x60300002d027 is located 0 bytes to the right of 23-byte region [0x60300002d010,0x60300002d027)
allocated by thread T0 here:
    #0 0x7f5154cbf91f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f5153e90676 in make_string /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/misc.c:129
    #2 0x7f5153e8b967 in PGAPI_ExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:157
    #3 0x7f5153ec83d0 in SQLExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:322
    #4 0x7f5154b94812 in SQLExecDirect (/lib/x86_64-linux-gnu/libodbc.so.2+0x16812)

SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827 in __interceptor_memcpy
Shadow bytes around the buggy address:
  0x0c067fffd9b0: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c067fffd9c0: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd
  0x0c067fffd9d0: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
  0x0c067fffd9e0: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
  0x0c067fffd9f0: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd
=>0x0c067fffda00: fa fa 00 00[07]fa fa fa fd fd fd fa fa fa fd fd
  0x0c067fffda10: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa
  0x0c067fffda20: fd fd fd fa fa fa fd fd fd fa fa fa 00 00 01 fa
  0x0c067fffda30: fa fa 00 00 02 fa fa fa 00 00 01 fa fa fa 00 00
  0x0c067fffda40: 03 fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c067fffda50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==52607==ABORTING

bookmark ................... ok   
declare-fetch-commit ....... ok   
declare-fetch-block ........ ok   
positioned-update .......... ok   
bulkoperations ............. 
=================================================================
==52634==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 640 byte(s) in 2 object(s) allocated from:
    #0 0x7f023c8bf91f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f023bc96990 in QR_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:184
    #2 0x7f023bc7193d in CC_send_query_append /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/connection.c:1950
    #3 0x7f023bcba98e in SC_execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2049
    #4 0x7f023bc8c626 in Exec_with_parameters_resolved /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:518
    #5 0x7f023bc8e612 in PGAPI_Execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:1182
    #6 0x7f023bc8bb65 in PGAPI_ExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:191
    #7 0x7f023bcc83d0 in SQLExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:322
    #8 0x7f023cfec812 in SQLExecDirect (/lib/x86_64-linux-gnu/libodbc.so.2+0x16812)

Indirect leak of 18 byte(s) in 2 object(s) allocated from:
    #0 0x7f023c873668 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:439
    #1 0x7f023bc970e5 in QR_set_command /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:368
    #2 0x7f023bc983e2 in QR_from_PGresult /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:709
    #3 0x7f023bc70759 in CC_from_PGresult /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/connection.c:1665
    #4 0x7f023bc72397 in CC_send_query_append /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/connection.c:2147
    #5 0x7f023bcba98e in SC_execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2049
    #6 0x7f023bc8c626 in Exec_with_parameters_resolved /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:518
    #7 0x7f023bc8e612 in PGAPI_Execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:1182
    #8 0x7f023bc8bb65 in PGAPI_ExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:191
    #9 0x7f023bcc83d0 in SQLExecDirect /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:322
    #10 0x7f023cfec812 in SQLExecDirect (/lib/x86_64-linux-gnu/libodbc.so.2+0x16812)

SUMMARY: AddressSanitizer: 658 byte(s) leaked in 4 allocation(s).

catalogfunctions ........... ok   
bindcol .................... ok   
lfconversion ............... ok   
cte ........................ ok   
errors ..................... 
=================================================================
==52663==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 320 byte(s) in 1 object(s) allocated from:
    #0 0x7fccea8bf91f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7fcce9c96990 in QR_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:184
    #2 0x7fcce9cbe654 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2953
    #3 0x7fcce9c7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #4 0x7fcce9c7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #5 0x7fcce9cbd97c in libpq_bind_and_exec /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2685
    #6 0x7fcce9cba718 in SC_execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2015
    #7 0x7fcce9c8c626 in Exec_with_parameters_resolved /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:518
    #8 0x7fcce9c8e612 in PGAPI_Execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:1182
    #9 0x7fcce9cc84f9 in SQLExecute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:352
    #10 0x7fcceafdb87e in SQLExecute (/lib/x86_64-linux-gnu/libodbc.so.2+0x1187e)

Indirect leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x7fccea8bf91f in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7fcce9c6b715 in CI_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/columninfo.c:30
    #2 0x7fcce9c969c5 in QR_Constructor /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/qresult.c:195
    #3 0x7fcce9cbe654 in ParseAndDescribeWithLibpq /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2953
    #4 0x7fcce9c7b70a in desc_params_and_sync /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2839
    #5 0x7fcce9c7b932 in prepareParameters /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/convert.c:2899
    #6 0x7fcce9cbd97c in libpq_bind_and_exec /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2685
    #7 0x7fcce9cba718 in SC_execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/statement.c:2015
    #8 0x7fcce9c8c626 in Exec_with_parameters_resolved /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:518
    #9 0x7fcce9c8e612 in PGAPI_Execute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/execute.c:1182
    #10 0x7fcce9cc84f9 in SQLExecute /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi.c:352
    #11 0x7fcceafdb87e in SQLExecute (/lib/x86_64-linux-gnu/libodbc.so.2+0x1187e)

SUMMARY: AddressSanitizer: 336 byte(s) leaked in 2 allocation(s).

error-rollback ............. ok   
diagnostic ................. ok   
numeric .................... ok   
large-object ............... ok   
large-object-data-at-exec .. ok   
odbc-escapes ............... 
=================================================================
==52705==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 10 byte(s) in 5 object(s) allocated from:
    #0 0x7f987b673668 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:439
    #1 0x7f987aad1afc in IPDSetField /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/pgapi30.c:940
    #2 0x7f987aad42a7 in PGAPI_SetDescField /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/pgapi30.c:1899
    #3 0x7f987aaceb10 in SQLSetDescField /home/progmachine/proj/workspaces/xenlab.workspace/psqlodbc/odbcapi30.c:448
    #4 0x7f987bd01ce8 in SQLSetDescField (/lib/x86_64-linux-gnu/libodbc.so.2+0x2bce8)

SUMMARY: AddressSanitizer: 10 byte(s) leaked in 5 allocation(s).

wchar-char ................. ok   
params-batch-exec .......... ok   
fetch-refcursors ........... ok   

[davecramer]

I've had a look, and I think they all have the same root cause which is the CC_clear_col_info function. It might be interesting to run the leak test with destroy set to true always to confirm

[progmachine]

It might be interesting to run the leak test with destroy set to true always to confirm

It's dangerous, because somewhere else will be dangling pointers. As a minimum, i know that StatementClass objects have same pointers, and if you always delete these objects, then use after free situations are possible.

[davecramer]

Well that would be good to know as well I would think.

[progmachine]

I think it is easy to reproduce:

1. Create two statements within one connection.
2. With first statement execute select command, and get Columns info with SQLDescribeCol function.
3. With second statement execute ALTER TABLE/DROP TABLE sql command. This will call CC_clear_col_info to clear COL_INFO cache.
4. Check again column info with first statement, that should access cached COL_INFO object, that was deleted on step 3 because of your modification.

But without AddressSanitizer, it can work well, until freed memory used again before step 4.

[progmachine]

I just now fixed memory leaks in premature test =) All 8 reported leaks was caused by a single error. Continuing working...

[davecramer]

PR ?

[progmachine]

PR ?

May be later? I want to fix all bugs reported in test suite, before creating PR.

[progmachine]

cursor-name, cursor-block-delete, bulkoperations tests is fixed. errors, odbc-escapes are next. Continuing working...

[progmachine]

All tests fixed. Leak of ci->password, ci->conn_settings, ci->pqopt in dlg_specific.c, lines 627, 673, 678 fixed. Only COL_INFO cache leak is left. Continuing working...

[progmachine]

Fixed them all, created PR #25.

[progmachine]

As PR was merged, this issue is done. Closing it.