Open aaronjwhiteside opened 6 years ago
I have also encountered the same issue in the demo https://docs.confluent.io/current/tutorials/cp-demo/docs/index.html#schema-registry-and-rest-proxy
It's not an issue with docker containers. I recreated this with version Kafka version : 1.1.0-cp1
directly on my laptop.
Here are my steps:
"auto.offset.reset": "earliest"
We're experiencing the same issue: first-time call to "consume" returns no records, a second call gets the records as expected.
This has only been tested with "auto.offet.reset" set to "earliest". The container logs for the REST proxy seem to show that the consumer gets connected to a broker on the first call, and then returns data on successive calls.
[2018-05-31 09:22:28,317] INFO [Consumer clientId=consumer-23, groupId=testing_consumation8] (Re-)joining group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:336)
Setup: a fresh install in a kubernetes cluster (based on https://github.com/Yolean/kubernetes-kafka), kafka v 1.1, rest proxy v 3.2.2 or 3.3 .
I am also experiencing this issue. Any suggestions to what the fix is?
Upgrade the kafka to 5.3.0 confluent kafka version. The issue is not observed in this kafka release with kafka rest proxy.
Based on some limited testing, it seems that this issue may occur if the consumer instance & subscription are created after producing to the topic. On the other hand, if the consumer instance & subscription are created before producing to the topic, then a single consume command appears to work.
We are on version 6.0.1 and this behavior is still happening. Exactly the same thing as ybyzek described above. (Sep 20, 2020)
We just did some testing on our own and found out that:
The following is the "on the wire" interaction between our REST client and the Kafka REST Proxy.
A summary of the operations performed is:
We have tried creating and subscribing the consumer before producing the message then trying to consume, but it makes no difference, in all cases we have tried we need to try and consume the topic twice to receive the message on the second attempt.
I can provide the output from all three docker containers if required.
Using the official docker images and the docker-maven-plugin to start them as part of a functional test. The behavior was observed with both versions 4.1.0 and 4.0.0.