artembilan
commented
2 years ago I don't see a behavior you are describing:
@SpringBootApplication
public class KafkaGh2302Application {
public static void main(String[] args) {
SpringApplication.run(KafkaGh2302Application.class, args);
}
@Bean
public NewTopic topic() {
return new NewTopic("seekExample", 3, (short) 1);
}
@Component
public static class Listener extends AbstractConsumerSeekAware {
@KafkaListener(id = "seekExample", topics = "seekExample", concurrency = "3")
public void listen(String payload) {
System.out.println("Listener received: " + payload);
}
public void seekToStart() {
getSeekCallbacks().forEach((tp, callback) -> callback.seekToBeginning(tp.topic(), tp.partition()));
}
}
@Component
public static class Listener2 extends AbstractConsumerSeekAware {
@KafkaListener(id = "seekExample2", topics = "seekExample", concurrency = "3")
public void listen(String payload) {
System.out.println("Listener2 received: " + payload);
}
public void seekToStart() {
getSeekCallbacks().forEach((tp, callback) -> callback.seekToBeginning(tp.topic(), tp.partition()));
}
}
}As you see I have two @KafkaListener classes with different ids which is, essentially, pointing to different consumer groups.
In the unit test I do:
@SpringBootTest
@EmbeddedKafka(bootstrapServersProperty = "spring.kafka.bootstrap-servers")
@DirtiesContext
class KafkaGh2302ApplicationTests {
@Autowired
KafkaGh2302Application.Listener listener;
@Autowired
KafkaTemplate<String, String> template;
@Test
void contextLoads() throws InterruptedException {
for (int i = 0; i < 10; i++) {
this.template.send("seekExample", i % 3, "some_key", "test#" + i);
}
Thread.sleep(1000);
this.listener.seekToStart();
Thread.sleep(10000);
}
}So, after sending some data, I just call seekToStart() on one of the listener services.
The output is like this:
Listener2 received: test#2
Listener received: test#1
Listener2 received: test#0
Listener2 received: test#5
Listener received: test#2
Listener received: test#4
Listener received: test#5
Listener received: test#7
Listener2 received: test#1
Listener2 received: test#8
Listener received: test#0
Listener2 received: test#3
Listener2 received: test#4
Listener received: test#8
Listener2 received: test#6
Listener received: test#3
Listener2 received: test#9
Listener2 received: test#7
Listener received: test#6
Listener received: test#9
Listener received: test#0
Listener received: test#3
Listener received: test#6
Listener received: test#9
Listener received: test#2
Listener received: test#5
Listener received: test#1
Listener received: test#4
Listener received: test#7
Listener received: test#8This confirms that seeking really happens only in one consumer group and it does not effect other groups on the same topic.
garyrussell
bky373
sobychacko
Expected Behavior
We want to be able to seek offset for specific consumer group by using AbstractConsumerSeekAware.
Current Behavior
regarding to below implementation it is clear that we can seek offset for all assigned partitions in a topic regardless of different consumer group ids.
Context For our use case there might be more than one group instance which is assigned same partition in a topic. Below example might be useful to describe our case:
Is there any way to seek offset in a partition but only for specific group id?