IBM / sarama

Sarama is a Go library for Apache Kafka.
MIT License
11.57k stars 1.76k forks source link

Encounter two questions about calling ResetOffset #2431

Closed yitian108 closed 1 year ago

yitian108 commented 1 year ago
Versions

Please specify real version numbers or git SHAs, not just "Latest" since that changes fairly regularly.

Sarama Kafka Go GoLand
1.37 2.5.0 1.17.11 2021.3.3
Configuration

What configuration values are you using for Sarama and Kafka?

Consumer.Offsets.AutoCommit.Enable = false

Kafka Topics [k, kk] Kafka Clusters: 3 servers Topic k, with partitions[0,1,2] Topic kk, with partition [0]

There are two testings for interface ResetOffset :

  1. Reset offset for topic kk, partition 0, and start offset at 11
    session.ResetOffset("kk", 0, 11, "")
  2. Reset offset for topic kk, partition 0, and start offset at 11, and reset offset for topic k, partition 0, and start offset at 1318895
    session.ResetOffset("kk", 0, 11, "")
    session.ResetOffset("k", 0, 1318895, "")
For Testing 1 Console Output

slowtoreset

For Testing 2 Console Output

miss reset offset

Problem Description

For Testing 1, we may find the consume time almost 5 seconds with receive 9 messages (Just Receive, did not handle anything), why it so slow for reseting offset to consume ?

For Testing 2, we also find the same issue with Testing 1, what's more, we may find two message missing when reset to consume,
for topic kk, the message with offset 11 not to receive to consume;
for topic k, the message with offset 1318895 not to receive to consume, which both the beginning offset message that specificed.

github-actions[bot] commented 1 year ago

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.