Try `arro3` as custom FileIO in tests

apache / iceberg-python

Apache PyIceberg

https://py.iceberg.apache.org/

Apache License 2.0

383 stars 140 forks source link

Open kevinjqliu opened 1 month ago

kevinjqliu commented 1 month ago

https://github.com/kylebarron/arro3 "A minimal Python library for Apache Arrow, connecting to the Rust arrow crate"

Set py-io-impl and run tests Good litmus test to check how tightly coupled the codebase is to pyarrow

kevinjqliu commented 1 week ago

As of v0.3.0, arro3 does not support reading from different FS https://kylebarron.dev/arro3/latest/api/io/