search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.22k stars 437 forks source link

[CH] Fix exception cause by one child of UnionExec outputs `Array(Nothing)` while the other outputs `Array(String)` #7979

Closed taiyang-li closed 1 day ago

taiyang-li commented 3 days ago

Description

sql: d_7262_0.sql 

Job aborted due to stage failure: Task 3 in stage 24.0 failed 2 times, most recent failure: Lost task 3.1 in stage 24.0 (TID 1211) (sg-dn2821.bigdata.bigo.inner executor 482): org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Array sizes are too large: 3906601321518741304
0. Poco::Exception::Exception(String const&, int) @ 0x0000000013b8de99
1. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000067b17f9
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000666a76c
3. DB::Exception::Exception<unsigned long&>(int, FormatStringHelperImpl<std::type_identity<unsigned long&>::type>, unsigned long&) @ 0x00000000067930ab
4. DB::SerializationArray::deserializeBinaryBulkWithMultipleStreams(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>*) const @ 0x000000000e9d413e
5. local_engine::readNormalComplexData(DB::ReadBuffer&, COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, local_engine::NativeReader::ColumnParseUtil&) @ 0x0000000006c0f06a
6. local_engine::NativeReader::appendNextBlock(DB::Block&) @ 0x0000000006c0e8b0
7. local_engine::NativeReader::read() @ 0x0000000006c0ce49
8. local_engine::ShuffleReader::read() @ 0x0000000006cf16e2
9. Java_org_apache_gluten_vectorized_CHStreamReader_nativeNext @ 0x0000000006656457

0. Poco::Exception::Exception(String const&, int) @ 0x0000000013b8de99
1. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000067b17f9
2. DB::Exception::createRuntime(int, String&) @ 0x0000000006668d0c
3. unsigned char local_engine::safeCallBooleanMethod<>(JNIEnv_*, _jobject*, _jmethodID*) @ 0x0000000006667d4b
4. local_engine::SourceFromJavaIter::peekBlock(JNIEnv_*, _jobject*) @ 0x0000000006c001c8
5. local_engine::ReadRelParser::parseReadRelWithJavaIter(substrait::ReadRel const&) @ 0x0000000006b9ad35
6. local_engine::ReadRelParser::parse(std::unique_ptr<DB::QueryPlan, std::default_delete<DB::QueryPlan>>, substrait::Rel const&, std::list<substrait::Rel const*, std::allocator<substrait::Rel const*>>&) @ 0x0000000006b9a6d0
7. local_engine::ReadRelParser::parse(std::vector<std::unique_ptr<DB::QueryPlan, std::default_delete<DB::QueryPlan>>, std::allocator<std::unique_ptr<DB::QueryPlan, std::default_delete<DB::QueryPlan>>>>&, substrait::Rel const&, std::list<substrait::Rel const*, std::allocator<substrait::Rel const*>>&) @ 0x0000000006b9bff6
8. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list<substrait::Rel const*, std::allocator<substrait::Rel const*>>&) @ 0x0000000006b785ee
9. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list<substrait::Rel const*, std::allocator<substrait::Rel const*>>&) @ 0x0000000006b77e59
10. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list<substrait::Rel const*, std::allocator<substrait::Rel const*>>&) @ 0x0000000006b77e59
11. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list<substrait::Rel const*, std::allocator<substrait::Rel const*>>&) @ 0x0000000006b77e59
12. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list<substrait::Rel const*, std::allocator<substrait::Rel const*>>&) @ 0x0000000006b77e59
13. local_engine::SerializedPlanParser::parseOp(substrait::Rel const&, std::list<substrait::Rel const*, std::allocator<substrait::Rel const*>>&) @ 0x0000000006b77e59
14. local_engine::SerializedPlanParser::parse(substrait::Plan const&) @ 0x0000000006b77b2f
15. local_engine::SerializedPlanParser::createExecutor(substrait::Plan const&) @ 0x0000000006b78fcf
16. Java_org_apache_gluten_vectorized_ExpressionEvaluatorJniWrapper_nativeCreateKernelWithIterator @ 0x000000000664fa1f