apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.64k stars 3.26k forks source link

[Bug] Be crash when select hive orc table wich has changed column type #19963

Open fsilent opened 1 year ago

fsilent commented 1 year ago

Search before asking

Version

1.2.3 master

What's Wrong?

reproduce:

Hive Side:

CREATE TABLE test_change3 (a int, b int, c int) stored as orc;

insert into table test_change3 values(1,8,9);

ALTER TABLE test_change3 CHANGE a a2 STRING;

Doris Side: refresh catalog hivecatalog; select * from test_change3; ----> be will crash

ERROR 1105 (HY000): RpcException, msg: org.apache.doris.rpc.RpcException: io.grpc.StatusRuntimeException: UNAVAILABLE: Network closed for unknown reason

be.out and coredump log:

0 0x000055749e00398b in construct<StringRef, char*&, long&> (__p=0x7f2da8a40490, this=) at /var/local/ldb-toolchain/include/c++/11/ext/new_allocator.h:154

1 construct<StringRef, char*&, long&> (p=0x7f2da8a40490, a=) at /var/local/ldb-toolchain/include/c++/11/bits/alloc_traits.h:512

2 emplace_back<char*&, long&> (this=) at /var/local/ldb-toolchain/include/c++/11/bits/vector.tcc:115

3 doris::vectorized::OrcReader::_decode_string_column (this=this@entry=0x7f2fd89dac00, col_name=..., data_column=..., type_kind=@0x7f2ec04930e0: orc::INT, cvb=cvb@entry=0x7f2da8ab1080,

num_values=1) at /data/52095938/be/src/vec/exec/format/orc/vorc_reader.cpp:643

4 0x000055749e00e614 in doris::vectorized::OrcReader::_orc_column_to_doris_column (this=this@entry=0x7f2fd89dac00, col_name=..., doris_column=..., data_type=...,

orc_column_type=0x7f2da89db540, cvb=0x7f2da8ab1080, num_values=1) at /data/52095938/be/src/vec/exec/format/orc/vorc_reader.cpp:738

5 0x000055749e0115b8 in doris::vectorized::OrcReader::get_next_block (this=0x7f2fd89dac00, block=0x7f2e88cdd5c0, read_rows=0x7f2ec0493318, eof=)

at /var/local/ldb-toolchain/include/c++/11/bits/stl_vector.h:1043

6 0x000055749dfd1ce4 in doris::vectorized::VFileScanner::_get_block_impl (this=0x7f2e88cf4000, state=, block=0x7f2e88cdd5c0, eof=0x7f2ec049353a)

at /var/local/ldb-toolchain/include/c++/11/bits/unique_ptr.h:421

7 0x000055749df9d799 in doris::vectorized::VScanner::get_block (this=this@entry=0x7f2e88cf4000, state=state@entry=0x7f2e504dd700, block=block@entry=0x7f2e88cdd5c0, eof=eof@entry=0x7f2ec049353a)

at /data/52095938/be/src/vec/exec/scan/vscanner.cpp:57

8 0x000055749df9ab23 in doris::vectorized::ScannerScheduler::_scanner_scan (this=, scheduler=, ctx=0x7f2e88cab000, scanner=0x7f2e88cf4000)

at /data/52095938/be/src/vec/exec/scan/scanner_scheduler.cpp:247

9 0x00005574992e9405 in operator() (this=) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:556

10 run (this=) at /data/52095938/be/src/util/threadpool.cpp:46

11 doris::ThreadPool::dispatch_thread (this=0x7f2fb0cf6c00) at /data/52095938/be/src/util/threadpool.cpp:535

12 0x00005574992deb3f in operator() (this=0x7f2fb08213f8) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:556

13 doris::Thread::supervise_thread (arg=0x7f2fb08213e0) at /data/52095938/be/src/util/thread.cpp:454

14 0x00007f2fda0a9ea5 in start_thread () from /lib64/libpthread.so.0

15 0x00007f2fda3bcb0d in clone () from /lib64/libc.so.6

What You Expected?

be not crash

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

Code of Conduct

zy-kkk commented 1 year ago

please upgrade 1.2.4.1

fsilent commented 1 year ago

please upgrade 1.2.4.1

can you tell me which issue number reslove this problem? I tested in master branch also has this problem.

fsilent commented 1 year ago

Unfortunately, after tested this problem also occurs in 1.2.4.1. Same crash @zy-kkk

zy-kkk commented 1 year ago

Unfortunately, after tested this problem also occurs in 1.2.4.1. Same crash @zy-kkk

Yes, thanks for providing a great example, I found it to be a bug, I will fix it

fandowe commented 1 year ago

Unfortunately, after tested this problem also occurs in 1.2.4.1. Same crash @zy-kkk

Yes, thanks for providing a great example, I found it to be a bug, I will fix it

你好,请问1.2.5版本对这个问题进行解决了嘛?

fsilent commented 1 year ago

execute insert overwrite table test_change select * from test_change; after change schema can avoid this problem