[HUDI-8433] Fix not update issuedOffset when stream read empty commits

apache / hudi

Upserts, Deletes And Incremental Processing on Big Data.

https://hudi.apache.org/

Apache License 2.0

5.46k stars 2.43k forks source link

[HUDI-8433] Fix not update issuedOffset when stream read empty commits #12166

Closed fhan688 closed 2 weeks ago

fhan688 commented 1 month ago

Change Logs

When empty commits are allowed , if number of continuous empty commits exceeds 'read.commits.limit' , IssuedInstant will not be updated which will block stream read.

Impact

hudi-flink-datasource

Risk level (write none, low medium or high below)

low

Documentation Update

None

Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

hudi-bot commented 2 weeks ago

CI report:

f20d0e028c9e1461a0a70f4a7715d5fa1254b5d5 Azure: FAILURE

Bot commands

@hudi-bot supports the following commands: - `@hudi-bot run azure` re-run the last Azure build

danny0405 commented 2 weeks ago

The Azure failure is not related.

fhan688 commented 2 weeks ago

The Azure failure is not related.

got it. thank you so much for Teacher Danny's advice and is there anything esle to revise?