This PR solves the issue of the control topics not being able to join successfully on a cloud environment as the current timeout is set to 1 second and it's not enough.
Before this PR: cg-control-XXX consumers are not always joining on time and the connector fails
After this PR: new timeout allows consumers join the consumer group and sink connectors work without issues
How to reproduce the issue?
Create an AWS MSK cluster with 1 topic (a local cluster is not valid as it's local network and it will surely work)
Install kafka-ui or any tool to check consumers of the topic
Create an Iceberg connector through MSK Connect
Check logs in Cloudwatch or the enabled logs tool when creating the connector
Post a message after initialisation
Check if the message is consumed (logs) and saved in your target system
Note: The connector will be marked as Running on MSK even if it couldn't start properly with the result of not consuming messages.
This PR solves the issue of the control topics not being able to join successfully on a cloud environment as the current timeout is set to 1 second and it's not enough.
Before this PR:
cg-control-XXXconsumers are not always joining on time and the connector fails After this PR: new timeout allows consumers join the consumer group and sink connectors work without issuesHow to reproduce the issue?
Note: The connector will be marked as
Runningon MSK even if it couldn't start properly with the result of not consuming messages.