duckdb / pg_duckdb

DuckDB-powered Postgres for high performance apps & analytics.
MIT License
1.58k stars 54 forks source link

Server crashes when scanning partitioned table #409

Open dpxcc opened 1 day ago

dpxcc commented 1 day ago

What happens?

Server crashes when executing the script below to scan partitioned table

To Reproduce

SET duckdb.force_execution = ON;
CREATE TABLE t (a INT) PARTITION BY LIST (a);
CREATE TABLE s (b INT);
SELECT * FROM t JOIN s ON a = b; -- crash

backtrace

#0  __pthread_kill_implementation (threadid=281473108209920, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x0000ffffa30cf254 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  0x0000ffffa308a67c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x0000ffffa3077130 in __GI_abort () at ./stdlib/abort.c:79
#4  0x0000aaaabfa25098 in ExceptionalCondition (conditionName=0xaaaabfc5ec70 "false", fileName=0xaaaabfc5e270 "bufmgr.c", lineNumber=3933) at assert.c:66
#5  0x0000aaaabf7bedc0 in RelationGetNumberOfBlocksInFork (relation=0xffffa3037fe0, forkNum=MAIN_FORKNUM) at bufmgr.c:3933
#6  0x0000ffffa361bc5c in pgduckdb::HeapReaderGlobalState::HeapReaderGlobalState (rel=0xffffa3037fe0, this=0xffff70005a30) at include/pgduckdb/scan/heap_reader.hpp:18
#7  __gnu_cxx::new_allocator<pgduckdb::HeapReaderGlobalState>::construct<pgduckdb::HeapReaderGlobalState, RelationData*&> (__p=0xffff70005a30, this=<optimized out>) at /usr/include/c++/11/ext/new_allocator.h:162
#8  std::allocator_traits<std::allocator<pgduckdb::HeapReaderGlobalState> >::construct<pgduckdb::HeapReaderGlobalState, RelationData*&> (__p=0xffff70005a30, __a=...) at /usr/include/c++/11/bits/alloc_traits.h:516
#9  std::_Sp_counted_ptr_inplace<pgduckdb::HeapReaderGlobalState, std::allocator<pgduckdb::HeapReaderGlobalState>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<RelationData*&> (__a=..., this=0xffff70005a20) at /usr/include/c++/11/bits/shared_ptr_base.h:519
#10 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<pgduckdb::HeapReaderGlobalState, std::allocator<pgduckdb::HeapReaderGlobalState>, RelationData*&> (__a=..., __p=<optimized out>, this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:650
#11 std::__shared_ptr<pgduckdb::HeapReaderGlobalState, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<pgduckdb::HeapReaderGlobalState>, RelationData*&> (__tag=..., this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr_base.h:1342
#12 std::shared_ptr<pgduckdb::HeapReaderGlobalState>::shared_ptr<std::allocator<pgduckdb::HeapReaderGlobalState>, RelationData*&> (__tag=..., this=<optimized out>) at /usr/include/c++/11/bits/shared_ptr.h:409
#13 std::allocate_shared<pgduckdb::HeapReaderGlobalState, std::allocator<pgduckdb::HeapReaderGlobalState>, RelationData*&> (__a=...) at /usr/include/c++/11/bits/shared_ptr.h:863
#14 std::make_shared<pgduckdb::HeapReaderGlobalState, RelationData*&> () at /usr/include/c++/11/bits/shared_ptr.h:879
#15 duckdb::make_shared_ptr<pgduckdb::HeapReaderGlobalState, RelationData*&> () at third_party/duckdb/src/include/duckdb/common/helper.hpp:73
#16 pgduckdb::PostgresSeqScanGlobalState::PostgresSeqScanGlobalState (this=0xffff70001390, rel=0xffffa3037fe0, input=...) at src/scan/postgres_seq_scan.cpp:19
#17 0x0000ffffa361c054 in duckdb::make_uniq<pgduckdb::PostgresSeqScanGlobalState, RelationData*&, duckdb::TableFunctionInitInput&> () at third_party/duckdb/src/include/duckdb/common/helper.hpp:65
#18 pgduckdb::PostgresSeqScanFunction::PostgresSeqScanInitGlobal (context=..., input=...) at src/scan/postgres_seq_scan.cpp:74
#19 0x0000ffff9fa112f8 in duckdb::TableScanGlobalSourceState::TableScanGlobalSourceState(duckdb::ClientContext&, duckdb::PhysicalTableScan const&) () from /usr/local/pgsql/lib/libduckdb.so
#20 0x0000ffff9fa09d04 in duckdb::PhysicalTableScan::GetGlobalSourceState(duckdb::ClientContext&) const () from /usr/local/pgsql/lib/libduckdb.so
#21 0x0000ffff9fbcdbf4 in duckdb::Pipeline::ResetSource(bool) () from /usr/local/pgsql/lib/libduckdb.so
#22 0x0000ffff9fbcdef8 in duckdb::Pipeline::Reset() () from /usr/local/pgsql/lib/libduckdb.so
#23 0x0000ffff9fbdc438 in duckdb::Pipeline::Schedule(duckdb::shared_ptr<duckdb::Event, true>&) () from /usr/local/pgsql/lib/libduckdb.so
#24 0x0000ffff9fbdc52c in duckdb::PipelineEvent::Schedule() () from /usr/local/pgsql/lib/libduckdb.so
#25 0x0000ffff9fbccf38 in duckdb::Event::CompleteDependency() () from /usr/local/pgsql/lib/libduckdb.so
#26 0x0000ffff9fbccdf8 in duckdb::Event::Finish() () from /usr/local/pgsql/lib/libduckdb.so
#27 0x0000ffff9fbdffc0 in duckdb::PipelineInitializeTask::ExecuteTask(duckdb::TaskExecutionMode) () from /usr/local/pgsql/lib/libduckdb.so
#28 0x0000ffff9fbd11a4 in duckdb::ExecutorTask::Execute(duckdb::TaskExecutionMode) () from /usr/local/pgsql/lib/libduckdb.so
#29 0x0000ffff9fbdbaec in duckdb::TaskScheduler::ExecuteForever(std::atomic<bool>*) () from /usr/local/pgsql/lib/libduckdb.so
#30 0x0000ffffa2ad31fc in ?? () from /lib/aarch64-linux-gnu/libstdc++.so.6
#31 0x0000ffffa30cd5c8 in start_thread (arg=0x0) at ./nptl/pthread_create.c:442
#32 0x0000ffffa3135edc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79

OS:

Linux

pg_duckdb Version:

0.1.0

Postgres Version:

17.0

Hardware:

No response

Full Name:

Cheng Chen

Affiliation:

Mooncake Labs

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Not applicable - the reproduction does not require a data set

Did you include all code required to reproduce the issue?

Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?

dpxcc commented 1 day ago

Potentially related to https://github.com/duckdb/pg_duckdb/issues/19

Leo-XM-Zeng commented 23 hours ago

It seems that the latest version cannot reproduce this anomaly

[postgres@halo-centos8 pg_duckdb]$ psql
psql (16.4)
Type "help" for help.

postgres=# create extension pg_duckdb;
CREATE EXTENSION
postgres=# SET duckdb.force_execution = ON;
SET
postgres=# CREATE TABLE t (a INT) PARTITION BY LIST (a);
CREATE TABLE
postgres=# CREATE TABLE s (b INT);
CREATE TABLE
postgres=# SELECT * FROM t JOIN s ON a = b;
 a | b 
---+---
(0 rows)

postgres=# \dx
                                                   List of installed extensions
   Name    | Version |   Schema   |                                          Description                                          
-----------+---------+------------+-----------------------------------------------------------------------------------------------
 orafce    | 4.14    | tpcds1     | Functions and operators that emulate a subset of functions and packages from the Oracle RDBMS
 pg_duckdb | 0.2.0   | public     | DuckDB Embedded in Postgres
 pg_jieba  | 1.1.1   | public     | a parser for full-text search of Chinese
 plpgsql   | 1.0     | pg_catalog | PL/pgSQL procedural language
(4 rows)

postgres=# \q
[postgres@halo-centos8 pg_duckdb]$ git branch -v
* main               58718a7 Fix typo in NOTICE file (#407)
dpxcc commented 19 hours ago

You need to compile Postgres with --enable-cassert to reproduce