This PR passes in the consumer group metadata during offset commit for better zombie fencing. Because it makes use of the sink's internal Kafka consumer, we no longer need the separate consumer group used to manage offsets.
Don't merge this, this is meant to demonstrate a potential solution to improve zombie fencing. There are a few further items that can be cleaned up. Also it was only briefly tested. cc @fqaiser94
This PR passes in the consumer group metadata during offset commit for better zombie fencing. Because it makes use of the sink's internal Kafka consumer, we no longer need the separate consumer group used to manage offsets.
Don't merge this, this is meant to demonstrate a potential solution to improve zombie fencing. There are a few further items that can be cleaned up. Also it was only briefly tested. cc @fqaiser94