apache / doris-flink-connector

Flink Connector for Apache Doris
https://doris.apache.org/
Apache License 2.0
323 stars 222 forks source link

[Bug] when use doris source to read data from doris-2.0,the be will crash #172

Open liuyang185 opened 1 year ago

liuyang185 commented 1 year ago

Search before asking

Version

apache-doris-2.0-beta-bin-x64-noavx2 flink-doris-connector-1.17.0

What's Wrong?

when use DorisSourceFunction to query data: DataStreamSource<List<?>> listDataStreamSource = env.addSource(new DorisSourceFunction(new DorisStreamOptions(props), new SimpleListDeserializationSchema())); then, be.out will print erro log: I0808 09:57:44.648388 30942 fragment_mgr.cpp:745] Register query/load memory tracker, query/load id: bb343d2f65b046ae-a5e7eca8e02e1ede limit: 2.00 GB I0808 09:57:44.648418 30942 plan_fragment_executor.cpp:115] PlanFragmentExecutor::prepare|query_id=TUniqueId(hi=-4957270016049264978, lo=-6491960127771500834)|instance_id=TUniqueId(hi=6071672337474381689, lo=-9070538737182937440)|backend_num=0|pthread_id=139938785482496 I0808 09:57:44.648770 7034 fragment_mgr.cpp:521] PlanFragmentExecutor::_exec_actual|query_id=bb343d2f65b046ae-a5e7eca8e02e1ede|instance_id=5442e9d263b9e779-821ef91388ca6ea0|pthread_id=139944866322176 I0808 09:57:44.648802 7034 plan_fragment_executor.cpp:253] PlanFragmentExecutor::open|query_id=TUniqueId(hi=-4957270016049264978, lo=-6491960127771500834)|instance_id=TUniqueId(hi=6071672337474381689, lo=-9070538737182937440)|mem_limit=2.00 GB Query id: bb343d2f65b046ae-a5e7eca8e02e1ede Aborted at 1691459864 (unix time) try "date -d @1691459864" if you are using GNU date Current BE git commitID: afe6bb9 SIGSEGV address not mapped to object (@0x40) received by PID 6627 (TID 7034 OR 0x7f47740b9700) from PID 64; stack trace: 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t, void) at /root/src/doris/be/src/common/signal_handler.h:413 1# os::Linux::chained_handler(int, siginfo, void) in /home/software/jenkins/java/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so 2# JVM_handle_linux_signal in /home/software/jenkins/java/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so 3# signalHandler(int, siginfo, void) in /home/software/jenkins/java/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so 4# 0x00007F481A0D42F0 in /lib64/libc.so.6 5# memcpy at /root/src/doris/be/src/glibc-compatibility/memcpy/memcpy_x86_64.cpp:219 6# arrow::FixedSizeBinaryBuilder::AppendValues(unsigned char const, long, unsigned char const) in /home/data/apache-doris-2.0/be/lib/doris_be 7# doris::vectorized::DataTypeNumberSerDe<__int128>::write_column_to_arrow(doris::vectorized::IColumn const&, unsigned char const, arrow::ArrayBuilder, int, int) const at /root/src/doris/be/src/vec/data_types/serde/data_type_number_serde.cpp:86 8# doris::FromBlockConverter::convert(std::shared_ptr) at /root/src/doris/be/src/util/arrow/block_convertor.cpp:392 9# doris::convert_to_arrow_batch(doris::vectorized::Block const&, std::shared_ptr const&, arrow::MemoryPool, std::shared_ptr) in /home/data/apache-doris-2.0/be/lib/doris_be 10# doris::vectorized::MemoryScratchSink::send(doris::RuntimeState, doris::vectorized::Block, bool) at /root/src/doris/be/src/vec/sink/vmemory_scratch_sink.cpp:83 11# doris::PlanFragmentExecutor::open_vectorized_internal() in /home/data/apache-doris-2.0/be/lib/doris_be 12# doris::PlanFragmentExecutor::open() at /root/src/doris/be/src/runtime/plan_fragment_executor.cpp:273 13# doris::FragmentExecState::execute() at /root/src/doris/be/src/runtime/fragment_mgr.cpp:263 14# doris::FragmentMgr::_exec_actual(std::shared_ptr, std::function<void (doris::RuntimeState, doris::Status)> const&) at /root/src/doris/be/src/runtime/fragment_mgr.cpp:527 15# std::_Function_handler<void (), doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&, std::function<void (doris::RuntimeState, doris::Status)> const&)::$_0>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 16# doris::ThreadPool::dispatch_thread() in /home/data/apache-doris-2.0/be/lib/doris_be 17# doris::Thread::supervise_thread(void) at /root/src/doris/be/src/util/thread.cpp:466 18# start_thread in /lib64/libpthread.so.0 19# __clone in /lib64/libc.so.6 ./bin/start_be.sh: 行 308: 6627 段错误 (吐核)${LIMIT:+${LIMIT}} "${DORIS_HOME}/lib/doris_be" "$@" 2>&1 < /dev/null

What You Expected?

how to fix it?

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

gnehil commented 1 year ago

You can contact me via WeChat for further information. My WeChat account is gnehil489.

JNSimba commented 3 months ago

Is there an error in the client response?