h2oai / db-benchmark

reproducible benchmark of database-like ops
https://h2oai.github.io/db-benchmark
Mozilla Public License 2.0
322 stars 85 forks source link

ClickHouse server crash during groupby 1e9 k=2 #112

Closed jangorecki closed 4 years ago

jangorecki commented 4 years ago
Error on processing query: SELECT id3, sum(v1) AS v1, avg(v3) AS v3 FROM G2_1e9_2e0_0_0 GROUP BY id3;

Code: 32, e.displayText() = DB::Exception: Attempt to read after eof: while receiving packet from localhost:9000, Stack trace:

0. clickhouse-client(StackTrace::StackTrace()+0x30) [0x7f58f30]
1. clickhouse-client(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x25) [0x3c427b5]
2. clickhouse-client(DB::throwReadAfterEOF()+0x55) [0x37b18fd]
3. clickhouse-client() [0x39e85ee]
4. clickhouse-client(DB::Client::receiveAndProcessPacket()+0x36) [0x3cc6686]
5. clickhouse-client(DB::Client::receiveResult()+0x14c) [0x3cc773c]
6. clickhouse-client(DB::Client::processSingleQuery(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<DB::IAST>)+0x40a) [0x3cc8baa]
7. clickhouse-client(DB::Client::process(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x32f) [0x3cc955f]
8. clickhouse-client(DB::Client::mainImpl()+0x137e) [0x3ccc1ce]
9. clickhouse-client(DB::Client::main(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)+0x25) [0x3b97455]
10. clickhouse-client(Poco::Util::Application::run()+0x38) [0x79f19e8]
11. clickhouse-client(mainEntryClickHouseClient(int, char**)+0x47) [0x3cab8f7]
12. clickhouse-client(main+0x213) [0x3b8e633]
13. /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7faa3004e830]
14. clickhouse-client(_start+0x2a) [0x3c4060a]
 (version 19.13.1.11 (official build))
jangorecki commented 4 years ago

After upgrade to recent ClickHouse version 19.16.2.2 this issue is also valid for G1_1e9_1e2_0_0 data, it also crashes clickhouse server now for all 1e9 data size tests.

jangorecki commented 4 years ago

After upgrading clickhouse to recent version 20.1.4.14, problem reported in this issue occurs now during q8 (top 2 rows by group):

Error on processing query: SELECT id6, arrayJoin(arraySlice(arrayReverseSort(gro
upArray(v3)), 1, 2)) AS v3 FROM G2_1e9_2e0_0_0 GROUP BY id6;

Code: 32, e.displayText() = DB::Exception: Attempt to read after eof: while rece
iving packet from localhost:9000, Stack trace (when copying this message, always
 include the lines below):

0. 0xbc34d6c Poco::Exception::Exception(std::__1::basic_string<char, std::__1::c
har_traits<char>, std::__1::allocator<char> > const&, int)  in /usr/bin/clickhou
se
1. 0x4969210 DB::throwReadAfterEOF()  in /usr/bin/clickhouse
2. 0x4ff31a6 DB::readVarUInt(unsigned long&, DB::ReadBuffer&)  in /usr/bin/click
house
3. 0x9152a24 DB::Connection::receivePacket()  in /usr/bin/clickhouse
4. 0x50475c1 DB::Client::receiveAndProcessPacket()  in /usr/bin/clickhouse
5. 0x504869a DB::Client::receiveResult()  in /usr/bin/clickhouse
6. 0x5048f35 DB::Client::processSingleQuery(std::__1::basic_string<char, std::__
1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<D
B::IAST>)  in /usr/bin/clickhouse
7. 0x5049b66 DB::Client::process(std::__1::basic_string<char, std::__1::char_tra
its<char>, std::__1::allocator<char> > const&)  in /usr/bin/clickhouse
8. 0x504b27d DB::Client::nonInteractive()  in /usr/bin/clickhouse
9. 0x504c9a3 DB::Client::mainImpl()  in /usr/bin/clickhouse
10. 0x4ebd756 DB::Client::main(std::__1::vector<std::__1::basic_string<char, std
::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::
__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >
 > > const&)  in /usr/bin/clickhouse
11. 0x9d3b883 Poco::Util::Application::run()  in /usr/bin/clickhouse
12. 0x501f3a9 mainEntryClickHouseClient(int, char**)  in /usr/bin/clickhouse
13. 0x4eb08f9 main  in /usr/bin/clickhouse
14. 0x20830 __libc_start_main /build/glibc-LK5gWL/glibc-2.23/csu/../csu/libc-sta
rt.c:325 in /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.23.so
15. 0x4f13eba _start  in /usr/bin/clickhouse
 (version 20.1.4.14 (official build))# clickhouse/exec.sh: benchmark sql script 
for G2_1e9_2e0_0_0 terminated with error
# clickhouse/exec.sh: finishing, clickhouse server down, possibly crashed, could
 not truncate table G2_1e9_2e0_0_0
jangorecki commented 4 years ago

This server crash has been fixed with https://github.com/h2oai/db-benchmark/commit/f8ad13877d447ce167c0005579ed883fd1510cba when addressing #132