search

apache / hudi-rs

A native Rust library for Apache Hudi, with bindings into Python
https://hudi.apache.org/
Apache License 2.0
149 stars 30 forks source link

[Idea] Support OpenDAL #131

Open alberttwong opened 2 months ago

alberttwong commented 2 months ago

Is there an existing issue for this?

Description of the bug

OpenDAL offers a unified data access layer, empowering users to seamlessly and efficiently retrieve data from diverse storage services. Our goal is to deliver a comprehensive solution for any languages, methods, integrations, and services.

ParadeDB uses Apache OpenDAL to integrate with object stores. If we support OpenDAL, all databases that use OpenDAL will be able to use Hudi

Steps To Reproduce

n/a

Expected behavior

n/a

Screenshots / Logs

No response

Software information

n/a

Additional context

No response

alberttwong commented 2 months ago

https://github.com/apache/opendal/issues/5065

Xuanwo commented 2 months ago

Thank you for bringing this up. I'm happy to help.

kazdy commented 2 months ago

there's a crate that brings opendal integrations to object_store hudi-rs uses: https://crates.io/crates/object_store_opendal It may be worth considering this instead of replacing it with opendal completely?