apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.
https://hudi.apache.org/
Apache License 2.0
5.35k stars 2.42k forks source link

[SUPPORT] Ensure `hoodie.datasource.write.keygenerator.type` internally sets `hoodie.datasource.write.keygenerator.class` and respects the table config validation across all code flows #7291

Open bhasudha opened 1 year ago

bhasudha commented 1 year ago

Tips before filing an issue

Describe the problem you faced We have hoodie.datasource.write.keygenerator.type that can internally instantiate hoodie.datasource.write.keygenerator.class. However the table config validations check only for hoodie.datasource.write.keygenerator.class. This creates ambiguity as to which property will take precedence and what will be honored.

To Reproduce

Steps to reproduce the behavior:

Expected behavior

A clear and concise description of what you expected to happen.

Environment Description

Additional context

Add any other context about the problem here.

Stacktrace

Add the stacktrace of the error.

xushiyan commented 1 year ago

For @lokeshj1703 to look further and present some ideas for discussion. The plan here should be: 1) understand the usage difference btw these 2 similar configs throughout the codebase. 2) think of an approach to make 1 config taking precedence and the other as fallback and deprecated. 3) Once the approach can be confirmed, then can come up with a fix and introduce the changing usage in release notes.