CometNativeException: called `Option::unwrap()` on a `None` value

andygrove commented 1 month ago

Describe the bug

I am seeing this error when running TPC-H q10 @ sf=100 with the filters removed.

org.apache.comet.CometNativeException: called `Option::unwrap()` on a `None` value
        at comet::errors::init::{{closure}}(__internal__:0)
        at std::panicking::rust_panic_with_hook(__internal__:0)
        at std::panicking::begin_panic_handler::{{closure}}(__internal__:0)
        at std::sys_common::backtrace::__rust_end_short_backtrace(__internal__:0)
        at rust_begin_unwind(__internal__:0)
        at core::panicking::panic_fmt(__internal__:0)
        at core::panicking::panic(__internal__:0)
        at core::option::unwrap_failed(__internal__:0)
        at <arrow_array::array::dictionary_array::TypedDictionaryArray<K,V> as arrow_array::array::ArrayAccessor>::value(__internal__:0)
        at comet::execution::datafusion::shuffle_writer::append_columns(__internal__:0)
        at comet::execution::datafusion::shuffle_writer::PartitionBuffer::append_rows(__internal__:0)
        at comet::execution::datafusion::shuffle_writer::ShuffleRepartitioner::append_rows_to_partition::{{closure}}(__internal__:0)
        at comet::execution::datafusion::shuffle_writer::external_shuffle::{{closure}}(__internal__:0)
        at <datafusion_physical_plan::stream::RecordBatchStreamAdapter<S> as futures_core::stream::Stream>::poll_next(__internal__:0)
        at Java_org_apache_comet_Native_executePlan(__internal__:0)