h2oai / db-benchmark

reproducible benchmark of database-like ops
https://h2oai.github.io/db-benchmark
Mozilla Public License 2.0
321 stars 85 forks source link

polars join 1e8 with NAs errors #187

Closed jangorecki closed 3 years ago

jangorecki commented 3 years ago

for join 1e8 data with NAs the following error is observed

thread '<unnamed>' panicked at 'assertion failed: i < (self.bits.len() << 3)', /
github/home/.cargo/git/checkouts/arrow-3a9cfebb6b7b2bdc/b0b622b/rust/arrow/src/b
itmap.rs:56:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "./polars/join-polars.py", line 55, in <module>
    ans = x.join(small, on="id1")
  File "/home/jan/git/db-benchmark/polars/py-polars/lib/python3.6/site-packages/
pypolars/frame.py", line 780, in join
    out = self._df.join(df._df, left_on, right_on, how)
pyo3_runtime.PanicException: assertion failed: i < (self.bits.len() << 3)

data are successfully loaded and error happens on the first query