Integrate hudi-rs in AWS SDK for Pandas

apache / hudi-rs

A native Rust library for Apache Hudi, with bindings into Python

https://hudi.apache.org/

Apache License 2.0

142 stars 28 forks source link

Integrate hudi-rs in AWS SDK for Pandas #172

Open kazdy opened 6 days ago

kazdy commented 6 days ago

Description of the improvement

AWS supports Hudi in most of their data services, many users leverage AWS SDK for Pandas (formerly AWS DataWrangler) to handle their data. Since hudi-rs provides Python bindings, we can support reading Hudi tables using the forementioned SDK.

Expected behavior

provide method in AWS SDK for Pandas" ::read_hudi(path, storage_options) that allows users to read from hudi tables

Additional context

No response

kazdy commented 6 days ago

@xushiyan please assign me to this one