pola-rs / pyo3-polars

Plugins/extension for Polars
MIT License
232 stars 38 forks source link

`ArrowInvalid` when return a `PyDataFrame` to python #66

Open sun-rs opened 6 months ago

sun-rs commented 6 months ago

After update polars to 0.37 and pyo3-polars to 0.11.1, I find there would be an error when return PyResult<PyDataFrame> to Python:

thread '<unnamed>' panicked at /Users/sun/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-polars-0.11.1/src/lib.rs:169:49:
called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'pyarrow.lib.ArrowInvalid'>, value: ArrowInvalid("Invalid or unsupported format string: 'vu'"), traceback: Some(<traceback object at 0x1032dfcc0>) }

My code works fine with polars 0.36 and pyo3-polars 0.10.0, and I have run cargo clean and delete cargo.lock. I print the DataFrame in rust side which is below: WX20240201-174527@2x

I find no "vu" in the df.

jshinonome commented 6 months ago

I have a similar issue. I think it is related to the new string/binary series type, as they are no longer generic pyarrow data types.

E       pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'pyarrow.lib.ArrowInvalid'>, value: ArrowInvalid("Invalid or unsupported format string: 'vu'"), traceback: Some(<traceback object at 0x7f329c07c340>) }

panic at this line https://github.com/pola-rs/pyo3-polars/blob/7b894de1ba196f06bbac8344822079d6ce1a3527/pyo3-polars/src/lib.rs#L169

jshinonome commented 6 months ago

This issue seems to be fixed.