apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.
https://hudi.apache.org/
Apache License 2.0
5.45k stars 2.43k forks source link

[HUDI-8411] Fix unmerged reader to skip delete blocks and avoid NPE #12144

Closed jonvex closed 1 month ago

jonvex commented 1 month ago

Change Logs

reader context had default implementation of get engine delete representation as null. The unmerged reader does not have handling for that. When compared with the other unmerged reader implementations they never process delete blocks, so the unmerged buffer is now updated to skip delete blocks

Impact

Spark npe is avoided and should be a slight perf boost. Also 1 less reader context method which is good.

Risk level (write none, low medium or high below)

none

Documentation Update

N/A

Contributor's checklist

hudi-bot commented 1 month ago

CI report:

Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build