The MirrorCheckpointConnector uses the global tasks max for setting the number of maximum tasks. This can cause extreme burden on single Kafka broker when hundreds of checkpoint tasks are consuming from from the single partition topic.
More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.
Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.
Committer Checklist (excluded from commit message)
The MirrorCheckpointConnector uses the global tasks max for setting the number of maximum tasks. This can cause extreme burden on single Kafka broker when hundreds of checkpoint tasks are consuming from from the single partition topic.
More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.
Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are expected for any behaviour change and system tests should be considered for larger changes.
Committer Checklist (excluded from commit message)