search

apache / hudi-rs

A native Rust library for Apache Hudi, with bindings into Python
https://hudi.apache.org/
Apache License 2.0
149 stars 30 forks source link

Upgrade arrow-rs, object_store, datafusion, pyo3 versions #143

Closed xushiyan closed 1 month ago

xushiyan commented 2 months ago

Upgrade the dependencies, and resolve any build issue.

kazdy commented 2 months ago

@xushiyan can you assign me to it please, that would be a good opportunity for me to get more familiar with the codebase

xushiyan commented 2 months ago

@kazdy there you go! do you think you can get this within these 2 days? trying to cut 0.2.0 branch within the week

kazdy commented 2 months ago

I'll try to deliver it asap

kazdy commented 2 months ago

@xushiyan seems like we should wait for datafusion to bump their dependencies, we're using the latest datafusion and it depends on arrow ^52.2.0 (so can't bump to 53.0), object_store ^0.10.2 so we can't bump it to 0.11.0, same goes for py3o.

Looks like it's best for the next datafusion release and upgrade these dependencies at that time. What do you think?

xushiyan commented 2 months ago

@kazdy sounds good. let's wait then.

xushiyan commented 1 month ago

@kazdy looks like there were some patch releases for these libs - would you be able to submit a PR to update to those patch versions? likely nothing will break - just getting some improvements for the next release :)

kazdy commented 1 month ago

@xushiyan ready in #168