[HUDI-8468] use payload to do merging inside the expression payload

apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.

https://hudi.apache.org/

Apache License 2.0

5.45k stars 2.43k forks source link

[HUDI-8468] use payload to do merging inside the expression payload #12201

Open jonvex opened 2 weeks ago

jonvex commented 2 weeks ago

Change Logs

Use payload to do merging inside of expression payload. If your tables payload was overwrite with latest, and you did MIT, it would use ordering field if defined. Now we will use the actual payload.

Impact

MIT is more correct

Risk level (write none, low medium or high below)

low

Documentation Update

N/A

Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

hudi-bot commented 2 weeks ago

CI report:

b2646a7e2b5277da5451d967ba12497598d2fdff UNKNOWN
ebed9d52c4028325d125c3742e8a6f288d6e2907 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build

Davis-Zhang-Onehouse commented 1 week ago

Folks, Jon is OOO and I'm taking care of this piece now. Let's review it here https://github.com/apache/hudi/pull/12242 as I cannot push to Jon's PR