apache / datafusion-python

Apache DataFusion Python Bindings

https://datafusion.apache.org/python

Apache License 2.0

329 stars 64 forks source link

Upgrade to DataFusion 30 #464

Closed andygrove closed 10 months ago

andygrove commented 11 months ago

Which issue does this PR close?

N/A

Rationale for this change

Next version

What changes are included in this PR?

Are there any user-facing changes?

andygrove commented 11 months ago

Some tests are failing:

FAILED datafusion/tests/test_dataframe.py::test_write_compressed_parquet[gzip-6] - AssertionError: assert 'uncompressed' == 'gzip'
FAILED datafusion/tests/test_dataframe.py::test_write_compressed_parquet[brotli-7] - AssertionError: assert 'uncompressed' == 'brotli'
FAILED datafusion/tests/test_dataframe.py::test_write_compressed_parquet[zstd-15] - AssertionError: assert 'uncompressed' == 'zstd'

mesejo commented 11 months ago

It seems that the issue was introduced by this commit on the main repo of datafusion. Currently the write_parquet method is not supporting the options.

andygrove commented 11 months ago

It seems that the issue was introduced by this commit on the main repo of datafusion. Currently the write_parquet method is not supporting the options.

Thanks for tracking that down! I filed https://github.com/apache/arrow-datafusion/issues/7433