search

apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.33k stars 3.21k forks source link

日期时间函数from_unixtime导致BE奔溃 #2472

Closed yuanfeng0905 closed 4 years ago

yuanfeng0905 commented 4 years ago

当执行下面 SQL 语句时，BE 节点奔溃：

SELECT website_id, order_id,from_unixtime(unix_timestamp(event_date_time)+28800, 'yyyy-MM-dd HH:mm:ss'),date_format(days_sub(now(), 7), '%Y-%m-%d 00:00:00')
    FROM ods__mysql__audit.logistics_track_item 
    WHERE website_id <> 1
    AND status_code = 'status_user_received'
    AND from_unixtime(unix_timestamp(event_date_time), 'yyyy-MM-dd HH:mm:ss')  >= null
limit 100

经测试发现， from_unixtime 函数与 null 比较大小均会导致 be 节点崩溃。

原始表结构为：

CREATE TABLE `logistics_track_item` (
  `website_id` int(11) NOT NULL COMMENT "",
  `entity_id` int(11) NOT NULL COMMENT "",
  `order_id` varchar(512) REPLACE NULL COMMENT "",
  `track_number` varchar(512) REPLACE NULL COMMENT "",
  `carriers_code` varchar(512) REPLACE NULL COMMENT "",
  `country` varchar(512) REPLACE NULL COMMENT "",
  `state` varchar(512) REPLACE NULL COMMENT "",
  `status_code` varchar(512) REPLACE NULL COMMENT "",
  `address` varchar(512) REPLACE NULL COMMENT "",
  `company_name` varchar(512) REPLACE NULL COMMENT "",
  `event_code` varchar(512) REPLACE NULL COMMENT "",
  `event_date_time` varchar(512) REPLACE NULL COMMENT "",
  `description` varchar(512) REPLACE NULL COMMENT "",
  `is_num` int(11) REPLACE NULL COMMENT "",
  `created_at` varchar(512) REPLACE NULL COMMENT "",
  `updated_at` varchar(512) REPLACE NULL COMMENT "",
  `dbus__action` varchar(512) REPLACE NULL COMMENT "",
  `dbus__timestamp` largeint(40) REPLACE NULL COMMENT ""
) ENGINE=OLAP
AGGREGATE KEY(`website_id`, `entity_id`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`entity_id`) BUCKETS 16
PROPERTIES (
"storage_type" = "COLUMN"
);

BE core dump 信息如下，(gdb bt): 

#0  0x00007f388028e5f7 in raise () from /lib64/libc.so.6
#1  0x00007f388028fce8 in abort () from /lib64/libc.so.6
#2  0x0000000002dc5765 in __gnu_cxx::__verbose_terminate_handler () at ../../.././libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x0000000002d31826 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../.././libstdc++-v3/libsupc++/eh_terminate.cc:47
#4  0x0000000002d31871 in std::terminate () at ../../.././libstdc++-v3/libsupc++/eh_terminate.cc:57
#5  0x0000000002d2ffd3 in __cxxabiv1::__cxa_throw (obj=obj@entry=0x21149dc40, tinfo=tinfo@entry=0x2efb600 <typeinfo for boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::gregorian::bad_year> >>,
    dest=dest@entry=0xdb9c60 <boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::gregorian::bad_year> >::~clone_impl()>) at ../../.././libstdc++-v3/libsupc++/eh_throw.cc:93
#6  0x0000000000dbdaae in boost::throw_exception<boost::gregorian::bad_year> (e=...) at /var/local/thirdparty/installed/include/boost/throw_exception.hpp:69
#7  0x0000000000dbdb3b in boost::CV::simple_exception_policy<unsigned short, (unsigned short)1400, (unsigned short)10000, boost::gregorian::bad_year>::on_error () at /var/local/thirdparty/installed/include/boost/date_time/constrained_value.hpp:110
#8  0x0000000000f6c8b2 in assign (value=13, this=<optimized out>) at /var/local/thirdparty/installed/include/boost/date_time/constrained_value.hpp:69
#9  constrained_value (value=13, this=<optimized out>) at /var/local/thirdparty/installed/include/boost/date_time/constrained_value.hpp:48
#10 greg_year (year=13, this=<optimized out>) at /var/local/thirdparty/installed/include/boost/date_time/gregorian/greg_year.hpp:42
#11 boost::date_time::parse_date<boost::gregorian::date> (s=..., order_spec=order_spec@entry=0) at /var/local/thirdparty/installed/include/boost/date_time/date_parsing.hpp:153
#12 0x0000000000f6a08f in parse_delimited_time<boost::posix_time::ptime> (sep=32 ' ', s=...) at /var/local/thirdparty/installed/include/boost/date_time/time_parsing.hpp:177
#13 time_from_string (s=...) at /var/local/thirdparty/installed/include/boost/date_time/posix_time/time_parsers.hpp:31
#14 doris::DateTimeValue::unix_timestamp (this=this@entry=0x7f3851fb4cb0, timestamp=timestamp@entry=0x7f3851fb4ca8, timezone=...) at /root/doris-0.11.25/be/src/runtime/datetime_value.cpp:1534
#15 0x0000000000db7557 in doris::TimestampFunctions::to_unix (context=<optimized out>, ts_val=...) at /root/doris-0.11.25/be/src/exprs/timestamp_functions.cpp:460
#16 0x0000000000e1d251 in doris::CastIntExpr::get_big_int_val (this=<optimized out>, context=<optimized out>, row=<optimized out>) at /root/doris-0.11.25/be/src/exprs/cast_expr.cpp:116
#17 0x0000000000df5b51 in doris::AddExpr::get_big_int_val (this=0x159900e00, context=0x18dc85a40, row=0x180346000) at /root/doris-0.11.25/be/src/exprs/arithmetic_expr.cpp:97
#18 0x0000000000e1d441 in doris::CastBigIntExpr::get_int_val (this=<optimized out>, context=<optimized out>, row=<optimized out>) at /root/doris-0.11.25/be/src/exprs/cast_expr.cpp:128
#19 0x0000000000d842fc in doris::ExprContext::get_value (this=this@entry=0x18dc85a40, e=<optimized out>, row=row@entry=0x180346000) at /root/doris-0.11.25/be/src/exprs/expr_context.cpp:296
#20 0x0000000000da0be0 in evaluate_children (input_vals=0x2122fec18, row=<optimized out>, context=0x18dc85a40, this=0x1abea7800) at /root/doris-0.11.25/be/src/exprs/scalar_fn_call.cpp:572
#21 doris::ScalarFnCall::interpret_eval<doris_udf::StringVal> (this=0x1abea7800, context=0x18dc85a40, row=<optimized out>) at /root/doris-0.11.25/be/src/exprs/scalar_fn_call.cpp:590
#22 0x0000000000e00987 in doris::GeStringValPred::get_boolean_val (this=0x169c38a80, ctx=0x18dc85a40, row=0x180346000) at /root/doris-0.11.25/be/src/exprs/binary_predicate.cpp:525
#23 0x00000000013b7663 in doris::ExecNode::eval_conjuncts (ctxs=<optimized out>, num_ctxs=<optimized out>, row=0x180346000) at /root/doris-0.11.25/be/src/exec/exec_node.cpp:508
#24 0x00000000014134f5 in doris::OlapScanner::get_batch (this=this@entry=0x214193200, state=<optimized out>, batch=batch@entry=0x19653c2c0, eof=eof@entry=0x7f3851fb4f67) at /root/doris-0.11.25/be/src/exec/olap_scanner.cpp:300
#25 0x00000000013ebf3d in doris::OlapScanNode::scanner_thread (this=0x1db10d000, scanner=0x214193200) at /root/doris-0.11.25/be/src/exec/olap_scan_node.cpp:1201
#26 0x0000000000f838a6 in operator() (this=0x7f3851fb5038) at /var/local/thirdparty/installed/include/boost/function/function_template.hpp:759
#27 doris::PriorityThreadPool::work_thread (this=0x148543b00, thread_id=<optimized out>) at /root/doris-0.11.25/be/src/util/priority_thread_pool.hpp:134
#28 0x00000000019f17ed in thread_proxy ()
#29 0x00007f3880044dc5 in start_thread () from /lib64/libpthread.so.0
#30 0x00007f388034f28d in clone () from /lib64/libc.so.6

doris 当前版本 ： v0.11.25

vagetablechicken commented 4 years ago

The root cause is time_from_string() parsing a invalid year, will throw exception Year is out of valid range: 1400..10000. v0.11.25 didn't catch it, so it cored. In this issue, year=13. We get this core in v0.11 too. It's fixed by https://github.com/apache/incubator-doris/pull/3198. This issue can be closed.

imay commented 4 years ago

@yuanfeng0905 You can upgrade your Doris, Thanks @vagetablechicken for your explaination, I will close this issue.