apache / hudi-rs

A native Rust library for Apache Hudi, with bindings into Python
https://hudi.apache.org/
Apache License 2.0
142 stars 28 forks source link

feat: add `TableBuilder` API for creating `Table` instances #163

Closed kazdy closed 6 days ago

kazdy commented 1 week ago

Description

Add TableBuilder to handle Table construction.

Allow to explicitly set hudi_options and storage_options via builder in rust or via python bindings when creating new Table instance.

let hudi_table = TableBuilder::from_base_uri("/tmp/hudi_table")
  .with_hudi_options([("hoodie.read.as.of.timestamp", "0")])
  .with_storage_options([("AWS_REGION", "us-east-1")])
  .build()

similarly for python

hudi_table = HudiTableBuilder.from_base_uri(
  "/tmp/hudi_table"
  ).with_hudi_options({
    "hoodie.read.as.of.timestamp": "0"
  }).with_storage_options({
    "AWS_REGION": "us-east-1"
  }).build()

resolves: #139

How are the changes test-covered

kazdy commented 1 week ago

@xushiyan please let me know if this is a direction you're fine with

xushiyan commented 1 week ago

@kazdy i'm cutting release branch within 1 day. do you think you can update this soon?

kazdy commented 1 week ago

@kazdy i'm cutting release branch within 1 day. do you think you can update this soon?

yes, I'll give it a try

xushiyan commented 1 week ago

@kazdy I'll push some fixes and then get this landed for release

codecov[bot] commented 1 week ago

Codecov Report

Attention: Patch coverage is 90.08264% with 12 lines in your changes missing coverage. Please review.

Project coverage is 90.35%. Comparing base (d5f42f6) to head (48821ab). Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
crates/core/src/table/builder.rs 88.76% 10 Missing :warning:
crates/core/src/table/mod.rs 71.42% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #163 +/- ## ========================================== + Coverage 89.81% 90.35% +0.53% ========================================== Files 17 19 +2 Lines 884 933 +49 ========================================== + Hits 794 843 +49 Misses 90 90 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.