At my company, our kafka consumer groups are restricted to a specific prefix, so the ability to override would be useful.
The issue
Currently there is a hardcoded group IcebergSinkConfig.DEFAULT_CONTROL_GROUP_PREFIX + UUID.randomUUID() that is raising an error on our cluster. The error is
[2023-12-21 03:46:15,972] ERROR [iceberg-cdc-datalake|task-0] WorkerSinkTask{id=iceberg-cdc-datalake-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:210)
org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: cg-control-0d46c80d-19c8-497d-9211-e6c17f1f9f66
Suggested fix
Add a configuration override for IcebergSinkConfig.DEFAULT_CONTROL_GROUP_PREFIX to allow for customization.
Background
At my company, our kafka consumer groups are restricted to a specific prefix, so the ability to override would be useful.
The issue
Currently there is a hardcoded group
IcebergSinkConfig.DEFAULT_CONTROL_GROUP_PREFIX + UUID.randomUUID()that is raising an error on our cluster. The error isSuggested fix
Add a configuration override for
IcebergSinkConfig.DEFAULT_CONTROL_GROUP_PREFIXto allow for customization.