search

scylladb / kafka-connect-scylladb

Kafka Connect Scylladb Sink
Apache License 2.0
42 stars 22 forks source link

Missing current-offset and lag when describing connector's consumer groups #68

Closed avelanarius closed 2 years ago

avelanarius commented 2 years ago

A problem reported by a user on Slack. When using kafka-consumer-group.sh to describe the consumer group of the Sink Connector, some of its details (current-offset and lag) are not correctly filled in:

Untitled

avelanarius commented 2 years ago

Issue successfully reproduced locally by me. I can confirm (as the user report stated) that the problem is present only when storing offsets on Scylla is disabled.

avelanarius commented 2 years ago

The only difference that could explain why it works when storing offsets on Scylla is enabled, are the code blocks guarded by if (config.isOffsetEnabledInScyllaDb()). There are 6 blocks like this, but only 2 look suspicious (both in ScyllaDbSinkTask). One that is probable to be the culprit is the one in preCommit. Maybe changing this one to call super.preCommit(...) on non-Scylla offset storage would fix the issue - to be further investigated.

avelanarius commented 2 years ago

Issue successfully reproduced locally by me.

Unfortunately, this was a premature celebration. Upon actually inserting some data into the replicated topic, the columns are correctly updated. @Bouncheck has similar problems to reproduce the issue.

canelmas commented 2 years ago

any idea/guidance on how to monitor for example the lag with this connector?

avelanarius commented 2 years ago

Upon further inspection, using kafka-consumer-group.sh is a good way to monitor the lag of the connector. We were unable to reproduce any issues with it and monitoring with kafka-consumer-group.sh works in both modes of storing offsets (both on Kafka and on Scylla).