search

apache / hudi-rs

A native Rust library for Apache Hudi, with bindings into Python
https://hudi.apache.org/
Apache License 2.0
149 stars 30 forks source link

Define Hudi error types across hudi-core #72

Open xushiyan opened 4 months ago

xushiyan commented 4 months ago

Use thiserror to define error enums to fit different error handling flows, and replace anyhow::Error usage.

xushiyan commented 3 months ago

@codope any plan to resume the work? Updated the milestone/release date.

gohalo commented 2 months ago

does this still wip? while i try to add some create table methods, just found this is a little chaos.

maybe i will try to refact that.

@codope @xushiyan

codope commented 2 months ago

@gohalo Please go ahead with your refactoring. I can help with review. I had opened https://github.com/apache/hudi-rs/pull/59 to refactor error handling but it is incomplete and preferably we shall use thiserror. You can open a separate PR, and I will close mine in your favour.

gohalo commented 2 months ago

Submit a pr, just replaced the anyhow package, but still have some problems to fix.

First, how to classified the errors.

  1. User configurations, invalid file/object path, unsupported features.
  2. Communication failed, mostly from the obejct_store crate.
  3. Invalid hoodie table, including file layout, file format, crashed data file and so on. Including from parquet arrow crates.
  4. and more.

Second, how to simplify the error generation or mapping process. thiserror provide many macros for this, such as error from etc. But still have many situation to simplify, through some trait like Into.