schema-file is required for hudi 0.9 to run mor compation

apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.

https://hudi.apache.org/

Apache License 2.0

5.23k stars 2.39k forks source link

schema-file is required for hudi 0.9 to run mor compation #11564

Closed njalan closed 2 weeks ago

njalan commented 2 weeks ago

where schema-file is required for hudi 0.9 to run mor compation? If I run one separate compaction job asynchronously on the table is ingesting by spark streaming whether will cause any data lost? is there any special parameter to config?

ad1happy2go commented 2 weeks ago

@njalan Yes you need to configure the lock provider. https://hudi.apache.org/docs/concurrency_control/#enabling-multi-writing

ad1happy2go commented 2 weeks ago

@njalan Do you have any more questions/issues on this?

njalan commented 2 weeks ago