Integrate hudi-rs in AWS SDK for Pandas

apache / hudi-rs

A native Rust library for Apache Hudi, with bindings into Python

https://hudi.apache.org/

Apache License 2.0

149 stars 30 forks source link

Integrate hudi-rs in AWS SDK for Pandas #172

Open kazdy opened 1 month ago

kazdy commented 1 month ago

Description of the improvement

AWS supports Hudi in most of their data services, many users leverage AWS SDK for Pandas (formerly AWS DataWrangler) to handle their data. Since hudi-rs provides Python bindings, we can support reading Hudi tables using the forementioned SDK.

Expected behavior

provide method in AWS SDK for Pandas" ::read_hudi(path, storage_options) that allows users to read from hudi tables

Additional context

No response

kazdy commented 1 month ago

@xushiyan please assign me to this one