pola-rs / polars

Dataframes powered by a multithreaded, vectorized query engine, written in Rust
https://docs.pola.rs
Other
30.16k stars 1.94k forks source link

`test_delta_encoding_roundtrip` hypothesis test failure in CI #18610

Open MarcoGorelli opened 1 month ago

MarcoGorelli commented 1 month ago
You can add @seed(91691230697029663097472053918202727681) to this test or run pytest with --hypothesis-seed=91691230697029663097472053918202727681 to reproduce this failure.

traceback:

``` ________________________ test_delta_encoding_roundtrip _________________________ [gw3] linux -- Python 3.12.5 /home/runner/work/polars/polars/py-polars/.venv/bin/python3 tests/unit/io/test_parquet.py:1435: in test_delta_encoding_roundtrip df=dataframes( tests/unit/io/test_parquet.py:1465: in test_delta_encoding_roundtrip assert_frame_equal(pl.read_parquet(f), df) polars/_utils/deprecation.py:91: in wrapper return function(*args, **kwargs) polars/_utils/deprecation.py:91: in wrapper return function(*args, **kwargs) polars/io/parquet/functions.py:170: in read_parquet return _read_parquet_binary( polars/io/parquet/functions.py:257: in _read_parquet_binary pydf = PyDataFrame.read_parquet( E pyo3_runtime.PanicException: attempt to add with overflow ----------------------------- Captured stderr call ----------------------------- thread 'polars-3' panicked at crates/polars-parquet/src/arrow/read/deserialize/primitive/mod.rs:227:64: attempt to add with overflow stack backtrace: 0: rust_begin_unwind at /rustc/c6db1ca3c93ad69692a4c4b5542f26fda4bf3aec/library/std/src/panicking.rs:662:5 1: core::panicking::panic_fmt at /rustc/c6db1ca3c93ad69692a4c4b5542f26fda4bf3aec/library/core/src/panicking.rs:74:14 2: core::panicking::panic_const::panic_const_add_overflow at /rustc/c6db1ca3c93ad69692a4c4b5542f26fda4bf3aec/library/core/src/panicking.rs:181:21 3: as polars_parquet::parquet::encoding::delta_bitpacked::decoder::DeltaGatherer>::gather_constant::{{closure}} 4: core::iter::adapters::map::map_fold::{{closure}} 5: core::iter::traits::iterator::Iterator::fold 6: as core::iter::traits::iterator::Iterator>::fold 7: core::iter::traits::iterator::Iterator::for_each 8: alloc::vec::Vec::extend_trusted 9: as alloc::vec::spec_extend::SpecExtend>::spec_extend 10: as core::iter::traits::collect::Extend>::extend 11: as polars_parquet::parquet::encoding::delta_bitpacked::decoder::DeltaGatherer>::gather_constant 12: polars_parquet::parquet::encoding::delta_bitpacked::decoder::Decoder::gather_miniblock_n_into 13: polars_parquet::parquet::encoding::delta_bitpacked::decoder::Decoder::gather_block_n_into 14: polars_parquet::parquet::encoding::delta_bitpacked::decoder::Decoder::gather_n_into 15: as polars_parquet::arrow::read::deserialize::utils::StateTranslation>>::extend_from_state 16: polars_parquet::arrow::read::deserialize::utils::State::extend_from_state 17: polars_parquet::arrow::read::deserialize::utils::NestedDecoder::push_n_valids 18: as polars_parquet::arrow::read::deserialize::utils::BatchableCollector<(),::DecodedState>>::push_n 19: polars_parquet::arrow::read::deserialize::utils::BatchedCollector::skip_in_place 20: polars_parquet::arrow::read::deserialize::nested_utils::extend_offsets2 21: polars_parquet::arrow::read::deserialize::nested_utils::PageNestedDecoder::collect_n 22: polars_parquet::arrow::read::deserialize::nested::columns_to_iter_recursive 23: polars_parquet::arrow::read::deserialize::nested::columns_to_iter_recursive 24: polars_parquet::arrow::read::deserialize::columns_to_iter_recursive 25: polars_parquet::arrow::read::deserialize::column_iter_to_arrays 26: polars_io::parquet::read::mmap::to_deserializer 27: polars_io::parquet::read::read_impl::column_idx_to_series 28: polars_io::parquet::read::read_impl::rg_to_dfs_par_over_rg::{{closure}}::{{closure}}::{{closure}} 29: core::iter::adapters::map::map_try_fold::{{closure}} 30: core::iter::traits::iterator::Iterator::try_fold 31: as core::iter::traits::iterator::Iterator>::try_fold 32: as core::iter::traits::iterator::Iterator>::try_fold 33: as core::iter::traits::iterator::Iterator>::next 34: alloc::vec::Vec::extend_desugared 35: as alloc::vec::spec_extend::SpecExtend>::spec_extend 36: as alloc::vec::spec_from_iter_nested::SpecFromIterNested>::from_iter 37: as alloc::vec::spec_from_iter::SpecFromIter>::from_iter 38: as core::iter::traits::collect::FromIterator>::from_iter 39: as core::iter::traits::collect::FromIterator>>::from_iter::{{closure}} 40: core::iter::adapters::try_process 41: as core::iter::traits::collect::FromIterator>>::from_iter 42: core::iter::traits::iterator::Iterator::collect 43: polars_io::parquet::read::read_impl::rg_to_dfs_par_over_rg::{{closure}}::{{closure}} 44: core::ops::function::impls:: for &F>::call_mut 45: core::ops::function::impls:: for &mut F>::call_once 46: as core::iter::traits::iterator::Iterator>::next 47: as core::iter::traits::iterator::Iterator>::next 48: as core::iter::traits::iterator::Iterator>::next 49: as core::iter::traits::iterator::Iterator>::next 50: alloc::vec::Vec::extend_desugared 51: as alloc::vec::spec_extend::SpecExtend>::spec_extend 52: as core::iter::traits::collect::Extend>::extend 53: as rayon::iter::plumbing::Folder>::consume_iter 54: as rayon::iter::plumbing::Folder>>::consume_iter 55: as rayon::iter::plumbing::Folder>::consume_iter 56: as rayon::iter::plumbing::Folder>::consume_iter 57: rayon::iter::plumbing::Producer::fold_with 58: rayon::iter::plumbing::bridge_producer_consumer::helper 59: rayon::iter::plumbing::bridge_producer_consumer::helper::{{closure}} 60: rayon_core::join::join_context::call_b::{{closure}} 61: rayon_core::job::StackJob::run_inline 62: rayon_core::join::join_context::{{closure}} 63: rayon_core::registry::in_worker 64: rayon_core::join::join_context 65: rayon::iter::plumbing::bridge_producer_consumer::helper 66: rayon::iter::plumbing::bridge_producer_consumer::helper::{{closure}} 67: rayon_core::join::join_context::call_b::{{closure}} 68: rayon_core::job::StackJob::run_inline 69: rayon_core::join::join_context::{{closure}} 70: rayon_core::registry::in_worker 71: rayon_core::join::join_context 72: rayon::iter::plumbing::bridge_producer_consumer::helper 73: rayon::iter::plumbing::bridge_producer_consumer::helper::{{closure}} 74: rayon_core::join::join_context::call_a::{{closure}} 75: as core::ops::function::FnOnce<()>>::call_once 76: std::panicking::try::do_call 77: __rust_try 78: std::panic::catch_unwind 79: rayon_core::unwind::halt_unwinding 80: rayon_core::join::join_context::{{closure}} 81: rayon_core::registry::in_worker 82: rayon_core::join::join_context 83: rayon::iter::plumbing::bridge_producer_consumer::helper 84: rayon::iter::plumbing::bridge_producer_consumer::helper::{{closure}} 85: rayon_core::join::join_context::call_a::{{closure}} 86: as core::ops::function::FnOnce<()>>::call_once 87: std::panicking::try::do_call 88: __rust_try 89: std::panic::catch_unwind 90: rayon_core::unwind::halt_unwinding 91: rayon_core::join::join_context::{{closure}} 92: rayon_core::registry::in_worker note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ---------------------------------- Hypothesis ---------------------------------- You can add @seed(91691230697029663097472053918202727681) to this test or run pytest with --hypothesis-seed=91691230697029663097472053918202727681 to reproduce this failure. =========================== short test summary info ============================ FAILED tests/unit/io/test_parquet.py::test_delta_encoding_roundtrip - pyo3_runtime.PanicException: attempt to add with overflow ====== 1 failed, 11493 passed, 10 skipped, 3 xfailed in 186.38s (0:03:06) ====== ```
nameexhaustion commented 1 month ago

also seed @seed(232884686442133804137552309983683525546)