apache / datafusion-python

Apache DataFusion Python Bindings
https://datafusion.apache.org/python
Apache License 2.0
320 stars 63 forks source link

Upgrade Datafusion to v37.1.0 #669

Closed Michael-J-Ward closed 1 month ago

Michael-J-Ward commented 1 month ago

NOTE: While each commit is a self-contained "logical" change, the project needs all the commits to compile.

Which issue does this PR close?

Closes #663. (hopefully)

This is a cleaned up version of #662.

Importantly, the final failing test is not commented out, so there is 1 failing test.

The error and test case are highlighted in this commit from previous PR: https://github.com/apache/datafusion-python/pull/662/commits/ecff35793e918a3dbdaf9220670d1ca5892b48d3

What changes are included in this PR?

Are there any user-facing changes?

Much of src/functions.rs have been updated per the upstream epic: https://github.com/apache/datafusion/issues/9285

Michael-J-Ward commented 1 month ago

One follow-on that I'd like to do is parametrize test_array_functions().

I didn't want to alter the test-cases unnecessarily while doing the upgrade.

slyons commented 1 month ago

Note this also closes #656 , #655, #654, #652, #651, #650 and #649