data_block_group_by_hash.rs" panic.line=325 panic.column=22

[BohuTANG]

Search before asking

• [X] I had searched in the issues and found no similar issues.

Version

main

What's Wrong?

2022-04-13T07:23:59.018461Z  INFO databend_query::servers::mysql::mysql_interactive_worker: Normal query: SELECT created_at AS date, COUNT(*) AS count FROM user GROUP BY created_at
2022-04-13T07:23:59.043220Z ERROR common_tracing::panic_hook: panicked at 'called `Result::unwrap()` on an `Err` value: Code: 1006, displayText = Function number_byte_size argument must be numeric types, but got Date16.

   0: common_exception::exception_code::<impl common_exception::exception::ErrorCode>::BadArguments
             at common/exception/src/exception_code.rs:36:66
   1: common_datavalues::types::type_id::TypeID::numeric_byte_size
             at common/datavalues/src/types/type_id.rs:210:30
   2: common_datablocks::kernels::data_block_group_by_hash::HashMethodFixedKeys<T>::deserialize_group_columns::{{closure}}
             at common/datablocks/src/kernels/data_block_group_by_hash.rs:322:17
   3: core::iter::adapters::map::map_fold::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/adapters/map.rs:84:28
   4: core::iter::traits::iterator::Iterator::fold
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/traits/iterator.rs:2362:21
   5: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/adapters/map.rs:124:9
   6: <usize as core::iter::traits::accum::Sum>::sum
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/traits/accum.rs:42:17
   7: core::iter::traits::iterator::Iterator::sum
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/traits/iterator.rs:3295:9
   8: common_datablocks::kernels::data_block_group_by_hash::HashMethodFixedKeys<T>::deserialize_group_columns
             at common/datablocks/src/kernels/data_block_group_by_hash.rs:318:32
   9: <databend_query::pipelines::transforms::group_by::aggregator_groups_builder::FixedKeysGroupColumnsBuilder<T> as databend_query::pipelines::transforms::group_by::aggregator_groups_builder::GroupColumnsBuilder<T>>::finish
             at query/src/pipelines/transforms/group_by/aggregator_groups_builder.rs:66:9
  10: <databend_query::pipelines::new::processors::transforms::aggregator::aggregator_final::FinalAggregator<_,Method> as databend_query::pipelines::new::processors::transforms::transform_aggregator::Aggregator>::generate
             at query/src/pipelines/new/processors/transforms/aggregator/aggregator_final.rs:191:44
  11: databend_query::pipelines::new::processors::transforms::transform_aggregator::GenerateState<TAggregator>::generate
             at query/src/pipelines/new/processors/transforms/transform_aggregator.rs:361:29
  12: <databend_query::pipelines::new::processors::transforms::transform_aggregator::AggregatorTransform<TAggregator> as databend_query::pipelines::new::processors::processor::Processor>::process
             at query/src/pipelines/new/processors/transforms/transform_aggregator.rs:263:53
  13: databend_query::pipelines::new::processors::processor::ProcessorPtr::process
             at query/src/pipelines/new/processors/processor.rs:91:9
  14: databend_query::pipelines::new::executor::executor_worker_context::ExecutorWorkerContext::execute_sync_task
             at query/src/pipelines/new/executor/executor_worker_context.rs:81:9
  15: databend_query::pipelines::new::executor::executor_worker_context::ExecutorWorkerContext::execute_task
             at query/src/pipelines/new/executor/executor_worker_context.rs:71:46
  16: databend_query::pipelines::new::executor::pipeline_executor::PipelineExecutor::execute_single_thread
             at query/src/pipelines/new/executor/pipeline_executor.rs:123:45
  17: databend_query::pipelines::new::executor::pipeline_executor::PipelineExecutor::execute_threads::{{closure}}
             at query/src/pipelines/new/executor/pipeline_executor.rs:93:23
  18: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/sys_common/backtrace.rs:122:18
  19: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/thread/mod.rs:498:17
  20: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/panic/unwind_safe.rs:271:9
  21: std::panicking::try::do_call
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:492:40
  22: __rust_try
  23: std::panicking::try
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:456:19
  24: std::panic::catch_unwind
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panic.rs:137:14
  25: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/thread/mod.rs:497:30
  26: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/ops/function.rs:227:5
  27: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/alloc/src/boxed.rs:1861:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/alloc/src/boxed.rs:1861:9
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/sys/unix/thread.rs:108:17
  28: start_thread
             at ./nptl/./nptl/pthread_create.c:442:8
  29: clone3
             at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
', /home/bohu/github/datafuselabs/databend/common/datablocks/src/kernels/data_block_group_by_hash.rs:325:22 backtrace=   0: common_tracing::panic_hook::set_panic_hook::{{closure}}
             at common/tracing/src/panic_hook.rs:25:25
   1: std::panicking::rust_panic_with_hook
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:702:17
   2: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:588:13
   3: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/sys_common/backtrace.rs:138:18
   4: rust_begin_unwind
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:584:5
   5: core::panicking::panic_fmt
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/panicking.rs:143:14
   6: core::result::unwrap_failed
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/result.rs:1785:5
   7: core::result::Result<T,E>::unwrap
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/result.rs:1078:23
   8: common_datablocks::kernels::data_block_group_by_hash::HashMethodFixedKeys<T>::deserialize_group_columns::{{closure}}
             at common/datablocks/src/kernels/data_block_group_by_hash.rs:322:17
   9: core::iter::adapters::map::map_fold::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/adapters/map.rs:84:28
  10: core::iter::traits::iterator::Iterator::fold
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/traits/iterator.rs:2362:21
  11: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/adapters/map.rs:124:9
  12: <usize as core::iter::traits::accum::Sum>::sum
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/traits/accum.rs:42:17
  13: core::iter::traits::iterator::Iterator::sum
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/iter/traits/iterator.rs:3295:9
  14: common_datablocks::kernels::data_block_group_by_hash::HashMethodFixedKeys<T>::deserialize_group_columns
             at common/datablocks/src/kernels/data_block_group_by_hash.rs:318:32
  15: <databend_query::pipelines::transforms::group_by::aggregator_groups_builder::FixedKeysGroupColumnsBuilder<T> as databend_query::pipelines::transforms::group_by::aggregator_groups_builder::GroupColumnsBuilder<T>>::finish
             at query/src/pipelines/transforms/group_by/aggregator_groups_builder.rs:66:9
  16: <databend_query::pipelines::new::processors::transforms::aggregator::aggregator_final::FinalAggregator<_,Method> as databend_query::pipelines::new::processors::transforms::transform_aggregator::Aggregator>::generate
             at query/src/pipelines/new/processors/transforms/aggregator/aggregator_final.rs:191:44
  17: databend_query::pipelines::new::processors::transforms::transform_aggregator::GenerateState<TAggregator>::generate
             at query/src/pipelines/new/processors/transforms/transform_aggregator.rs:361:29
  18: <databend_query::pipelines::new::processors::transforms::transform_aggregator::AggregatorTransform<TAggregator> as databend_query::pipelines::new::processors::processor::Processor>::process
             at query/src/pipelines/new/processors/transforms/transform_aggregator.rs:263:53
  19: databend_query::pipelines::new::processors::processor::ProcessorPtr::process
             at query/src/pipelines/new/processors/processor.rs:91:9
  20: databend_query::pipelines::new::executor::executor_worker_context::ExecutorWorkerContext::execute_sync_task
             at query/src/pipelines/new/executor/executor_worker_context.rs:81:9
  21: databend_query::pipelines::new::executor::executor_worker_context::ExecutorWorkerContext::execute_task
             at query/src/pipelines/new/executor/executor_worker_context.rs:71:46
  22: databend_query::pipelines::new::executor::pipeline_executor::PipelineExecutor::execute_single_thread
             at query/src/pipelines/new/executor/pipeline_executor.rs:123:45
  23: databend_query::pipelines::new::executor::pipeline_executor::PipelineExecutor::execute_threads::{{closure}}
             at query/src/pipelines/new/executor/pipeline_executor.rs:93:23
  24: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/sys_common/backtrace.rs:122:18
  25: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/thread/mod.rs:498:17
  26: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/panic/unwind_safe.rs:271:9
  27: std::panicking::try::do_call
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:492:40
  28: __rust_try
  29: std::panicking::try
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:456:19
  30: std::panic::catch_unwind
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panic.rs:137:14
  31: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/thread/mod.rs:497:30
  32: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/ops/function.rs:227:5
  33: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/alloc/src/boxed.rs:1861:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/alloc/src/boxed.rs:1861:9
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/sys/unix/thread.rs:108:17
  34: start_thread
             at ./nptl/./nptl/pthread_create.c:442:8
  35: clone3
             at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
 panic.file="/home/bohu/github/datafuselabs/databend/common/datablocks/src/kernels/data_block_group_by_hash.rs" panic.line=325 panic.column=22
2022-04-13T07:23:59.046882Z ERROR databend_query::servers::mysql::writers::query_result_writer: OnQuery Error: Code: 1015, displayText = Any { .. }.

   0: common_exception::exception_code::<impl common_exception::exception::ErrorCode>::LogicalError
             at common/exception/src/exception_code.rs:36:66
   1: databend_query::pipelines::new::executor::pipeline_executor::PipelineExecutor::execute
             at query/src/pipelines/new/executor/pipeline_executor.rs:79:35
   2: databend_query::pipelines::new::executor::pipeline_pulling_executor::PipelinePullingExecutor::thread_function::{{closure}}
             at query/src/pipelines/new/executor/pipeline_pulling_executor.rs:96:33
   3: core::ops::function::FnOnce::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/ops/function.rs:227:5
   4: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/sys_common/backtrace.rs:122:18
   5: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/thread/mod.rs:498:17
   6: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/panic/unwind_safe.rs:271:9
   7: std::panicking::try::do_call
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:492:40
   8: __rust_try
   9: std::panicking::try
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panicking.rs:456:19
  10: std::panic::catch_unwind
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/panic.rs:137:14
  11: std::thread::Builder::spawn_unchecked_::{{closure}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/thread/mod.rs:497:30
  12: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/core/src/ops/function.rs:227:5
  13: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/alloc/src/boxed.rs:1861:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/alloc/src/boxed.rs:1861:9
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/76d770ac21d9521db6a92a48c7b3d5b2cc535941/library/std/src/sys/unix/thread.rs:108:17

How to Reproduce?

Create table:

CREATE TABLE `user` (
  `created_at` Date,
  `count` Int32,
) 

Insert into datas:

insert into user values('2022-04-01', 5)

Select:

mysql> SELECT created_at AS date, COUNT(*) AS count FROM user GROUP BY created_at;
ERROR 1105 (HY000): Code: 1015, displayText = Any { .. }.

If the server panic, client can get the correct ERROR info here:

ERROR 1105 (HY000): Code: 1015, displayText = Any { .. }.

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

[BohuTANG]

@sundy-li , is it a way to escape the error from the closure? https://github.com/datafuselabs/databend/blob/d4baa9c928284e8c491488519bd24d610421fc83/common/datablocks/src/kernels/data_block_group_by_hash.rs#L321-L325

Found some .numeric_byte_size().unwrap() in data_block_group_by_hash.rs, seems all of them is in a closure.

[sundy-li]

Yes, unwrap is too dangerous here.

[BohuTANG]

This works:

        let mut null_offsize = group_fields
            .iter()
            .map(|c| {
                let ty = c.data_type();
                remove_nullable(ty).data_type_id().numeric_byte_size()
            })
            .sum::<Result<usize>>()?;