search

StarRocks / starrocks

StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries.
https://starrocks.io
Apache License 2.0
8.77k stars 1.76k forks source link

BE crash randomly when query from HUDI external table #4548

Closed tiannan-sr closed 2 years ago

tiannan-sr commented 2 years ago

Steps to reproduce the behavior (Required)

  1. create hudi table with parquet file format and gzip compression, and insert data:

    
create table hudi_parquet_gzip(
uuid int,
col_boolean boolean,
col_int int,
col_long long,
col_float float,
col_double double,
col_decimal decimal(38,18),
col_date date,
col_timestamp timestamp,
col_string string,
col_binary binary,
col_array array<int>,
col_struct struct<a:string,b:int>,
col_map map<string,int>
) USING hudi 
TBLPROPERTIES(
type = 'cow',
primarykey = 'uuid');

insert into hudi_parquet_gzip select 1,true,1,1.01,1.001,1.0001,10000000000.0000000001,cast('2020-01-01' as date),cast('2020-01-01 00:00:01' as timestamp),'Top 10 Unsolved Mysteries of Paleontological Dinosaurs, Did You Know?',cast('1110001010101011001001' as binary),array(1,10,100),null,null;
insert into hudi_parquet_gzip select 2,false,2,2.02,2.002,2.0002,20000000000.0000000002,cast('2020-02-01' as date),cast('2020-02-01 00:00:02' as timestamp),'Xi Jinping, general secretary of the Communist Party of China Central Committee, called on Tuesday for unrelenting efforts in exercising full and rigorous governance over the Party, saying that the CPC will continue to show zero tolerance for corruption.',cast('1110001010101011001001' as binary),array(2,20,200),null,null;
insert into hudi_parquet_gzip select 3,true,3,3.03,3.003,3.0003,30000000000.0000000003,cast('2020-03-01' as date),cast('2020-03-01 00:00:04' as timestamp),'Nation raises caution on overseas packages',cast('1110001010101011001001' as binary),array(3,30,300),null,null;
insert into hudi_parquet_gzip select 4,false,4,4.04,4.004,4.0004,40000000000.0000000004,cast('2020-04-01' as date),cast('2020-04-01 00:00:04' as timestamp),'Shanghai, Shenzhen register most newly listed firms in 2021',cast('1110001010101011001001' as binary),array(4,40,400),null,null;
insert into hudi_parquet_gzip select 5,true,5,5.05,5.005,5.0005,50000000000.0000000005,cast('2020-05-01' as date),cast('2020-05-01 00:00:05' as timestamp),'A total of 4,685 companies have been listed on the A-share market in China as of Dec 31, 2021, with 46 percent of them based in Beijing, Shanghai, Shenzhen, Hangzhou, Suzhou, Guangzhou, Ningbo, Nanjing, Wuxi, and Chengdu, said a report of National Business Daily on Wednesday.',cast('1110001010101011001001' as binary),array(5,50,500),null,null;
insert into hudi_parquet_gzip select 6,false,6,6.06,6.006,6.0006,60000000000.0000000006,cast('2020-06-01' as date),cast('2020-06-01 00:00:06' as timestamp),'The development of enterprises, as well as the push of local governments, has boosted the listings, the report said. For example, in early 2018, Central China''s Hubei province launched a plan to double its listed companies by including the listing of enterprises into the government''s annual performance assessment.',cast('1110001010101011001001' as binary),array(6,60,600),null,null;
insert into hudi_parquet_gzip select 7,true,7,7.07,7.007,7.0007,70000000000.0000000007,cast('2020-07-01' as date),cast('2020-07-01 00:00:07' as timestamp),'Most listed companies in the top 10 cities come from industries such as information technology, electronics, mechanical equipment, medical biology, and electric power equipment, with information technology bearing most listed companies.',cast('1110001010101011001001' as binary),array(7,70,700),null,null;
insert into hudi_parquet_gzip select 8,false,8,8.08,8.008,8.0008,80000000000.0000000008,cast('2020-08-01' as date),cast('2020-08-01 00:00:08' as timestamp),'Citi report finds MNC mood in China buoyant',cast('1110001010101011001001' as binary),array(8,80,800),null,null;
insert into hudi_parquet_gzip select 9,true,9,9.09,9.009,9.0009,90000000000.0000000009,cast('2020-09-01' as date),cast('2020-09-01 00:00:09' as timestamp),'Xi: China, Russia major champions of multilateralism, global justice',cast('1110001010101011001001' as binary),array(9,90,900),null,null;
insert into hudi_parquet_gzip select 10,false,10,10.10,10.010,10.00010,100000000000.00000000010,cast('2020-10-01' as date),cast('2020-10-01 00:00:10' as timestamp),'Volkswagen deliveries hit by chip shortages',cast('1110001010101011001001' as binary),array(10,100,1000),null,null;
insert into hudi_parquet_gzip select 11,true,11,11.11,11.011,11.00011,110000000000.00000000011,cast('2020-11-01' as date),cast('2020-11-01 00:00:11' as timestamp),'Light installations featuring the tiger, the Chinese zodiac animal for the upcoming lunar year, will be set up in the main plaza.',cast('1110001010101011001001' as binary),array(11,110,1100),null,null;
insert into hudi_parquet_gzip select 12,false,12,12.12,12.012,12.00012,120000000000.00000000012,cast('2020-12-01' as date),cast('2020-10-01 00:00:10' as timestamp),'The annual lantern show at Yuyuan Garden, a historic tourist site in Shanghai, will kick off on Tuesday and run through Feb 28.',cast('1110001010101011001001' as binary),array(12,120,1200),null,null;
insert into hudi_parquet_gzip select 13,null,null,null,null,null,null,null,null,null,null,null,null,null;
insert into hudi_parquet_gzip select 14,null,null,null,null,null,12345678901234567890.123456789012345678,null,null,'',null,null,null,null;

2. create hudi external table:

create external resource hudi_resource_1648542744569 PROPERTIES("type"="hudi", "hive.metastore.uris"="thrift://..***.10:9083"); create external table ex_hudi_tbl0 ( uuid int ,col_boolean boolean ,col_int int ,col_long bigint ,col_float float ,col_double double ,col_decimal decimal(38,18) ,col_date date ,col_string string ,col_array array ,col_binary varchar(200)) ENGINE=hudi properties ( "resource" = "hudi_resource_1648542744569", "table" = "hudi_parquet_gzip", "database" = "hudi_db");


3. select from hudi external table , BE crash randomly:

mysql> select col_boolean,col_int,col_long,col_float,col_double,col_decimal,col_date,col_string,col_array,col_binary from ex_hudi_tbl0 order by col_int limit 5; +-------------+---------+----------+-----------+------------+-----------------------------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+------------------------+ | col_boolean | col_int | col_long | col_float | col_double | col_decimal | col_date | col_string | col_array | col_binary | +-------------+---------+----------+-----------+------------+-----------------------------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+------------------------+ | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | NULL | | NULL | NULL | NULL | NULL | NULL | 12345678901234567890.123456789012345678 | NULL | | NULL | NULL | | 1 | 1 | 1 | 1.001 | 1.0001 | 10000000000.000000000100000000 | 2020-01-01 | Top 10 Unsolved Mysteries of Paleontological Dinosaurs, Did You Know? | [1,10,100] | 1110001010101011001001 | | 0 | 2 | 2 | 2.002 | 2.0002 | 20000000000.000000000200000000 | 2020-02-01 | Xi Jinping, general secretary of the Communist Party of China Central Committee, called on Tuesday for unrelenting efforts in exercising full and rigorous governance over the Party, saying that the CPC will continue to show zero tolerance for corruption. | [2,20,200] | 1110001010101011001001 | | 1 | 3 | 3 | 3.003 | 3.0003 | 30000000000.000000000300000000 | 2020-03-01 | Nation raises caution on overseas packages | [3,30,300] | 1110001010101011001001 | +-------------+---------+----------+-----------+------------+-----------------------------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+------------------------+ 5 rows in set (0.03 sec)

mysql> select col_boolean,col_int,col_long,col_float,col_double,col_decimal,col_date,col_string,col_array,col_binary from ex_hudi_tbl0 order by col_int limit 5; ^[[A^[[DERROR 1064 (HY000): Ocurrs time out with specfied time 299997 MILLISECONDS mysql> show backends; +-----------+-----------------+---------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------------------------------------------------------+------------------------+--------------------------------------------------------+-------------------+-------------+ | BackendId | Cluster | IP | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime | LastHeartbeat | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | ErrMsg | Version | Status | DataTotalCapacity | DataUsedPct | +-----------+-----------------+---------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------------------------------------------------------+------------------------+--------------------------------------------------------+-------------------+-------------+ | 511273 | default_cluster | 172.26.92.141 | 9454 | 9460 | 8438 | 8460 | 2022-03-29 16:45:04 | 2022-03-29 16:45:54 | false | false | false | 2552 | 2.363 GB | 408.822 GB | 1.968 TB | 79.72 % | 79.72 % | java.net.ConnectException: Connection refused (Connection refused) | QA_TEST_MASTER-61b1f1f | {"lastSuccessReportTabletsTime":"2022-03-29 16:45:04"} | 411.185 GB | 0.57 % | | 10004 | default_cluster | 172.26.92.151 | 9454 | 9460 | 8438 | 8460 | 2022-03-29 16:45:19 | 2022-03-29 16:53:24 | true | false | false | 2539 | 2.348 GB | 605.309 GB | 1.968 TB | 69.97 % | 69.97 % | | QA_TEST_MASTER-61b1f1f | {"lastSuccessReportTabletsTime":"2022-03-29 16:53:19"} | 607.656 GB | 0.39 % | | 10002 | default_cluster | 172.26.92.152 | 9454 | 9460 | 8438 | 8460 | 2022-03-29 16:45:29 | 2022-03-29 16:53:24 | true | false | false | 2587 | 2.366 GB | 758.161 GB | 1.968 TB | 62.39 % | 62.39 % | | QA_TEST_MASTER-61b1f1f | {"lastSuccessReportTabletsTime":"2022-03-29 16:52:29"} | 760.527 GB | 0.31 % | +-----------+-----------------+---------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------------------------------------------------------+------------------------+--------------------------------------------------------+-------------------+-------------+ 3 rows in set (0.00 sec)


### Expected behavior (Required)
query return right result

### Real behavior (Required)
BE crash:

terminate called after throwing an instance of 'std::length_error' what(): vector::_M_range_insert Aborted at 1648543557 (unix time) try "date -d @1648543557" if you are using GNU date PC: @ 0x7feaf2524387 GI_raise SIGABRT (@0x3f700000bcf) received by PID 3023 (TID 0x7fea83524700) from PID 3023; stack trace: @ 0x7feaf2fd9630 (unknown) @ 0x7feaf2524387 __GI_raise @ 0x7feaf2525a78 GI_abort @ 0x186aeb1 _ZN9gnu_cxx27verbose_terminate_handlerEv.cold @ 0x5699f86 cxxabiv1::terminate() @ 0x5699ff1 std::terminate() @ 0x569a144 cxa_throw @ 0x186ca95 std::throw_length_error() @ 0x19b0a20 starrocks::vectorized::FixedLengthColumnBase<>::append() @ 0x2375e7f starrocks::vectorized::NullableColumn::append() @ 0x2360dc2 starrocks::vectorized::ArrayColumn::append() @ 0x2375e96 starrocks::vectorized::NullableColumn::append() @ 0x236b7ab starrocks::vectorized::Chunk::append_safe() @ 0x259c5bf starrocks::vectorized::HeapChunkSorter::done() @ 0x25963d4 starrocks::vectorized::TopNNode::_consume_chunks() @ 0x25978f0 starrocks::vectorized::TopNNode::open() @ 0x1f3a11b starrocks::PlanFragmentExecutor::_open_internal_vectorized() @ 0x1f3ae07 starrocks::PlanFragmentExecutor::open() @ 0x1ec7632 starrocks::FragmentExecState::execute() @ 0x1ecbe4c starrocks::FragmentMgr::exec_actual() @ 0x1ecc681 _ZNSt17_Function_handlerIFvvEZN9starrocks11FragmentMgr18exec_plan_fragmentERKNS1_23TExecPlanFragmentParamsERKSt8functionIFvPNS1_20PlanFragmentExecutorEEESC_EUlvE_E9_M_invokeERKSt9_Any_data @ 0x2008499 starrocks::ThreadPool::dispatch_thread() @ 0x2004058 starrocks::Thread::supervise_thread() @ 0x7feaf2fd1ea5 start_thread @ 0x7feaf25ec8dd __clone @ 0x0 (unknown)


### StarRocks version (Required)
 - You can get the StarRocks version by executing SQL `select current_version()`

mysql> select current_version(); +------------------------+ | current_version() | +------------------------+ | QA_TEST_MASTER 61b1f1f | +------------------------+ 1 row in set (0.01 sec)

miomiocat commented 2 years ago

i will fix it

tiannan-sr commented 2 years ago

Using the latest 2.2 branch code, the problem cannot be reproduced, close the issue.