search

datafuselabs / databend

š——š—®š˜š—®, š—”š—»š—®š—¹š˜†š˜š—¶š—°š˜€ & š—”š—œ. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
https://docs.databend.com
Other
7.67k stars 726 forks source link

Bug: panices on inserting data from an internal stage. #15656

Open flaneur2020 opened 3 months ago

flaneur2020 commented 3 months ago

Summary

Hi we are having an issue at the moment inserting data from an internal stage. Cloud dropping and creating an internal stage once every minute cause an issue?

May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]: panicked at src/common/arrow/src/arrow/bitmap/bitmap_ops.rs:119:5:
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]: assertion `left == right` failed
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:   left: 343
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:  right: 902
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    0: std::backtrace_rs::backtrace::libunwind::trace
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/../../backtrace/src/backt>
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    1: std::backtrace_rs::backtrace::trace_unsynchronized
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/../../backtrace/src/backt>
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    2: std::backtrace::Backtrace::create
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/backtrace.rs:331:13
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    3: databend_common_tracing::panic_hook::log_panic
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./workspace/src/common/tracing/src/panic_hook.rs:35:21
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    4: databend_common_tracing::panic_hook::set_panic_hook::{{closure}}
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./workspace/src/common/tracing/src/panic_hook.rs:30:9
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    5: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/alloc/src/boxed.rs:2030:9
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    6: std::panicking::rust_panic_with_hook
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:785:13
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    7: std::panicking::begin_panic_handler::{{closure}}
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:659:13
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    8: std::sys_common::backtrace::__rust_end_short_backtrace
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/sys_common/backtrace.rs:1>
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:    9: rust_begin_unwind
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/std/src/panicking.rs:647:5
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:   10: core::panicking::panic_fmt
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/core/src/panicking.rs:72:14
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:   11: core::panicking::assert_failed_inner
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:   12: core::panicking::assert_failed
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./rustc/d5fd0997291ca0135401a39dff25c8a9c13b8961/library/core/src/panicking.rs:297:5
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:   13: databend_common_arrow::arrow::bitmap::bitmap_ops::binary
May 28 06:02:09 ip-10-0-45-39.ec2.internal databend-query[8970]:              at ./workspace/src/common/arrow/src/arrow/bitmap/bitmap_ops.rs:119:5

the version is v1.2.452

sundy-li commented 3 months ago

Any reproduced sql ?

sundy-li commented 3 months ago 

INSERT INTO raw.bsc_transactions (access_list, ...52 fields in total)
SELECT source.* 
    FROM (
            SELECT * FROM @staged_table  -- this matches the above 52 fields
        )source
    LEFT ANTI JOIN(
        SELECT hash, block_timestamp FROM raw.bsc_transactions
        WHERE block_timestamp >= '2024-05-15 23:13:48.000000' AND block_timestamp <= '2024-05-16 00:16:32.000000'
    ) destination
    ON destination.hash = source.hash

From user: I have been testing locally, and if i where to the source query to have only some rows from the staged table then it inserts fine, but the full insert query was a problem