Our company kafka cluster requires specific consumer groups prefixes so the default ones need to be configurable.
The issue
When overriding iceberg.connect.group-id, the sink stops working and can no longer consume data. If you remove this configuration, the sink will work. This can be repro'd on docker with the following configuration:
Background
Our company kafka cluster requires specific consumer groups prefixes so the default ones need to be configurable.
The issue
When overriding iceberg.connect.group-id, the sink stops working and can no longer consume data. If you remove this configuration, the sink will work. This can be repro'd on docker with the following configuration:
If I remove
iceberg.connect.group-id
and issue an update request, the commit loop starts working.The code looks to have implemented an override for this, so I am unsure why it doesn't work - see https://github.com/tabular-io/iceberg-kafka-connect/blob/4c4b8f8408846926545ae453cfe4ffbdae7801cd/kafka-connect/src/main/java/io/tabular/iceberg/connect/IcebergSinkConfig.java#L404