finos / tracdap

A next-generation data and analytics platform for use in highly regulated environments
Apache License 2.0
43 stars 12 forks source link

Revert Arrow 14 Upgrade #405

Closed martin-traverse closed 1 year ago

martin-traverse commented 1 year ago

There is a regression for AWS S3 stroage in Apache Arrow 14:

https://github.com/apache/arrow/issues/38618

We need to hold off upgrading until this is addressed, or implement a workaround.

This change also rolls back support for Python 3.12. Arrow 14 comes with a wheel for 3.12 where Arrow 13 needs to run build wheel tools (these will fail on many machines that don't have all the extra build tools available).