search

apache / arrow-rs

Official Rust implementation of Apache Arrow
https://arrow.apache.org/
Apache License 2.0
2.63k stars 807 forks source link

Separate pyo3 arrow integration into standalone crate #6776

Open xmakro opened 1 week ago

xmakro commented 1 week ago

The pyo3 crate released version 0.23. The arrow crate depends on pyo3 0.22 and it is not possible for arrow to update the pyo3 dependency without a major breaking release. Therefore, it is currently not possible for a user to upgrade a codebase that depends both on arrow and pyo3.

To fix this, we could instead move the arrow pyo3 integration into a separate crate. This crate could then be updated to a new version of pyo3 outside of the arrow major release cadence. Since it is a separate crate, it would require wrapper structs around structs that implement IntoPyObject.

tustvold commented 5 days ago

The next arrow release, due in December will be breaking and incorporate this version upgrade. We make breaking releases at most every 3 months, looking at the history of pyo3 breaking releases, this shouldn't lead to that much of a lag.