search

delta-io / delta-rs

A native Rust library for Delta Lake, with bindings into Python
https://delta-io.github.io/delta-rs/
Apache License 2.0
1.97k stars 365 forks source link

feat(#2597): allow pyarrow.dataset.Expression in filters kwarg #2600

Closed giacomorebecchi closed 1 week ago

giacomorebecchi commented 2 weeks ago

Description

Allow pyarrow.dataset.Expression in filters kwarg of deltalake.DeltaTable.to_pyarrow_table and deltalake.DeltaTable.to_pandas methods, and remove code dupllication in favor of the pyarrow implementation.

Related Issue(s)

giacomorebecchi commented 2 weeks ago

@ion-elgreco fixed both errors that were spotted in the actions run, sorry for overlooking them!

roeap commented 2 weeks ago

looks like we need to add a few type hints to the pyarrow pyi files (in this repo) for linting to pass.

giacomorebecchi commented 1 week ago

looks like we need to add a few type hints to the pyarrow pyi files (in this repo) for linting to pass.

Please let me know if my last commit works, thanks for your patience!