Fix conversion from arrow column to arrow RecordBatch - Githubissues

mars-project / mars

Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.

https://mars-project.readthedocs.io

Apache License 2.0

2.68k stars 325 forks source link

Fix conversion from arrow column to arrow RecordBatch #3359

Closed wjsi closed 9 months ago

wjsi commented 9 months ago

What do these changes do?

Conversions from pandas DataFrames containing ArrowArrays to arrow RecordBatch is now fixed. Also fix some compatibility issues due to dependency updates.

Related issue number

Fixes #xxxx

Check code requirements

[x] tests added / passed (if needed)
[x] Ensure all linting tests pass, see here for how to run them