duckdb / pg_duckdb

DuckDB-powered Postgres for high performance apps & analytics.
MIT License
1.62k stars 57 forks source link

Server crashes if parallel run "type_support" test #425

Open saygoodbyye opened 1 week ago

saygoodbyye commented 1 week ago

What happens?

If we parallel run type_support test like below, server will crash PostgreSQL build:

CFLAGS='-Og' ./configure --enable-tap-tests --with-openssl --enable-debug --enable-cassert --with-icu --with-lz4 --with-libxml

To Reproduce

Patch test/regression/schedule:

test: type_support type_support type_support type_support type_support

Then execute

make installcheck

regression.out:

# parallel group (5 tests):  type_support type_support type_support type_support type_support
not ok 1     + type_support                              533 ms
# (test process exited with exit code 2)
not ok 2     + type_support                              741 ms
# (test process exited with exit code 2)
not ok 3     + type_support                              533 ms
# (test process exited with exit code 2)
not ok 4     + type_support                              533 ms
# (test process exited with exit code 2)
not ok 5     + type_support                              533 ms
# (test process exited with exit code 2)

backtrace:

#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007df5cd64526e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007df5cd6288ff in __GI_abort () at ./stdlib/abort.c:79
#5  0x000059942390782a in ExceptionalCondition (conditionName=conditionName@entry=0x59942396c470 "TransactionIdFollowsOrEquals(xid, TransactionXmin)", fileName=fileName@entry=0x5994239fe068 "subtrans.c", lineNumber=lineNumber@entry=169) at assert.c:66
#6  0x000059942346c579 in SubTransGetTopmostTransaction (xid=xid@entry=892) at subtrans.c:169
#7  0x000059942394a7fb in XidInMVCCSnapshot (xid=892, snapshot=snapshot@entry=0x599425328030) at snapmgr.c:1929
#8  0x0000599423422a0e in HeapTupleSatisfiesMVCC (htup=<optimized out>, snapshot=0x599425328030, buffer=1866) at heapam_visibility.c:1071
#9  0x0000599423423d1a in HeapTupleSatisfiesVisibility (htup=<optimized out>, snapshot=<optimized out>, buffer=<optimized out>) at heapam_visibility.c:1772
#10 0x00007df5ce2e086a in pgduckdb::HeapReader::ReadPageTuples (this=0x7df590001130, output=...) at src/scan/heap_reader.cpp:126
#11 0x00007df5ce2e497d in pgduckdb::PostgresSeqScanFunction::PostgresSeqScanFunc (context=..., data=..., output=...) at src/scan/postgres_seq_scan.cpp:102
#12 0x00007df5c9fbc49e in duckdb::PhysicalTableScan::GetData(duckdb::ExecutionContext&, duckdb::DataChunk&, duckdb::OperatorSourceInput&) const () from /tmp/pgsql/lib/libduckdb.so
#13 0x00007df5ca160a0f in duckdb::PipelineExecutor::FetchFromSource(duckdb::DataChunk&) () from /tmp/pgsql/lib/libduckdb.so
#14 0x00007df5ca16bfe9 in duckdb::PipelineExecutor::Execute(unsigned long) () from /tmp/pgsql/lib/libduckdb.so
#15 0x00007df5ca16c352 in duckdb::PipelineTask::ExecuteTask(duckdb::TaskExecutionMode) () from /tmp/pgsql/lib/libduckdb.so
#16 0x00007df5ca16258d in duckdb::ExecutorTask::Execute(duckdb::TaskExecutionMode) () from /tmp/pgsql/lib/libduckdb.so
#17 0x00007df5ca16b04b in duckdb::TaskScheduler::ExecuteForever(std::atomic<bool>*) () from /tmp/pgsql/lib/libduckdb.so
#18 0x00007df5cd2ecdb4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x00007df5cd69ca94 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#20 0x00007df5cd729c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

OS:

Ubuntu 24.04

pg_duckdb Version:

main

Postgres Version:

REL_17_STABLE

Hardware:

No response

Full Name:

Egor Chindyaskin

Affiliation:

Postgres Professional

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a source build

Did you include all relevant data sets for reproducing the issue?

Yes

Did you include all code required to reproduce the issue?

Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?

mkaruza commented 6 days ago

@saygoodbyye hi, can you provide commit hash was used for this bug?

saygoodbyye commented 6 days ago

@mkaruza, 992853852554d1da2ab87b41ff8d21d21abab776

saygoodbyye commented 5 days ago

@mkaruza, I have pulled new changes and tried to execute new array_type_support test added in #282 like above. So I've got the same assert. regression.out

# parallel group (5 tests):  array_type_support array_type_support array_type_support array_type_support array_type_support
not ok 8     + array_type_support                        934 ms
# (test process exited with exit code 2)
not ok 9     + array_type_support                        937 ms
# (test process exited with exit code 2)
not ok 10    + array_type_support                        939 ms
# (test process exited with exit code 2)
not ok 11    + array_type_support                        937 ms
# (test process exited with exit code 2)
not ok 12    + array_type_support                        936 ms
# (test process exited with exit code 2)

backtrace:

#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007c0006c4526e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007c0006c288ff in __GI_abort () at ./stdlib/abort.c:79
#5  0x000060319379482a in ExceptionalCondition (conditionName=conditionName@entry=0x6031937f9470 "TransactionIdFollowsOrEquals(xid, TransactionXmin)", fileName=fileName@entry=0x60319388b068 "subtrans.c", lineNumber=lineNumber@entry=169) at assert.c:66
#6  0x00006031932f9579 in SubTransGetTopmostTransaction (xid=xid@entry=1018) at subtrans.c:169
#7  0x00006031937d77fb in XidInMVCCSnapshot (xid=1018, snapshot=snapshot@entry=0x603195051090) at snapmgr.c:1929
#8  0x00006031932afa0e in HeapTupleSatisfiesMVCC (htup=<optimized out>, snapshot=0x603195051090, buffer=2219) at heapam_visibility.c:1071
#9  0x00006031932b0d1a in HeapTupleSatisfiesVisibility (htup=<optimized out>, snapshot=<optimized out>, buffer=<optimized out>) at heapam_visibility.c:1772
#10 0x00007c00077c74db in pgduckdb::HeapReader::ReadPageTuples (this=0x7bfed0000f50, output=...) at /usr/include/c++/13/bits/unique_ptr.h:199
#11 0x00007c00077cb5ed in pgduckdb::PostgresSeqScanFunction::PostgresSeqScanFunc (context=..., data=..., output=...) at src/scan/postgres_seq_scan.cpp:101
#12 0x00007c00035bc49e in duckdb::PhysicalTableScan::GetData(duckdb::ExecutionContext&, duckdb::DataChunk&, duckdb::OperatorSourceInput&) const () from /tmp/pgsql/lib/libduckdb.so
#13 0x00007c0003760a0f in duckdb::PipelineExecutor::FetchFromSource(duckdb::DataChunk&) () from /tmp/pgsql/lib/libduckdb.so
#14 0x00007c000376bfe9 in duckdb::PipelineExecutor::Execute(unsigned long) () from /tmp/pgsql/lib/libduckdb.so
#15 0x00007c000376c352 in duckdb::PipelineTask::ExecuteTask(duckdb::TaskExecutionMode) () from /tmp/pgsql/lib/libduckdb.so
#16 0x00007c000376258d in duckdb::ExecutorTask::Execute(duckdb::TaskExecutionMode) () from /tmp/pgsql/lib/libduckdb.so
#17 0x00007c000376b04b in duckdb::TaskScheduler::ExecuteForever(std::atomic<bool>*) () from /tmp/pgsql/lib/libduckdb.so
#18 0x00007c00068ecdb4 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#19 0x00007c0006c9ca94 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#20 0x00007c0006d29c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Moreover, I get warnings while building:

src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<16> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:638:40:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
  542 |         auto &child_type = GetChildTypeRecursive(value.type());
      |               ^~~~~~~~~~
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<18> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:642:40:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<21> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:646:40:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<23> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:650:40:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<20> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:654:40:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<1043> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:661:43:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<1082> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:665:40:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<1114> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:669:45:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<700> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:673:42:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<701> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:677:42:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<1700> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:681:43:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]
src/pgduckdb_types.cpp: In instantiation of ‘void pgduckdb::ConvertDuckToPostgresArray(TupleTableSlot*, duckdb::Value&, idx_t) [with OP = PODArray<PostgresOIDMapping<2950> >; TupleTableSlot = TupleTableSlot; idx_t = long unsigned int]’:
src/pgduckdb_types.cpp:685:40:   required from here
src/pgduckdb_types.cpp:542:15: warning: unused variable ‘child_type’ [-Wunused-variable]