Closed ruihe774 closed 3 weeks ago
I'm still wondering how to ensure forward compatibility with future=True
. When a old version of pyo3-polars
is used together with a newer version of python polars which has introduced incompatible arrow data structure updates, it can be broken. One solution is to check the version of polars before setting future=True
, and another solution is to introduce versioning like future=1
.
As for backward compatibility, we can simply retry with future=True
unset if it fails.
I've made this PR work with it. In PySeries::into_py
and PySeries::extract_bound
, newest compatibility level supported by both sides is negotiated and selected automatically. In this way, both backward compatibility and forward compatibility are achieved.
This depends on https://github.com/pola-rs/polars/pull/17371.
In my naive benchmark, with large string columns, it improves the speed to ~10x.