[HUDI-8519] Fix update with multiple secondary indexes

lokeshj1703 commented 6 days ago

Change Logs

Multiple secondary indexes (or functional index) exist in different partitions but still we use the same file id prefix. So, thre is a chance of collision in the append handle when two different secondary index have same file id prefix and same shard. This PR fixes the file id prefix in such a case. Added updates to the existing test case which creates multiple secondary index.

Impact

Fix updates with multiple secondary indexes.

Risk level (write none, low medium or high below)

low only affects sec index and func index.

Documentation Update

NA

Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

lokeshj1703 commented 6 days ago

The PR builds on top of https://github.com/apache/hudi/pull/12259 and addresses review comments

lokeshj1703 commented 6 days ago

@nsivabalan I have addressed your comments. PTAL.

hudi-bot commented 6 days ago

CI report:

22bd5bc4dc4d32c2db79e6c1343413fb9830a794 UNKNOWN
1a0c9f5916be95a1a70b62deb4be670e261a2016 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build

apache / hudi