confluentinc / kafka-rest

Confluent REST Proxy for Kafka
https://docs.confluent.io/current/kafka-rest/docs/index.html
Other
33 stars 640 forks source link

Commiting a list of offsets does not work #536

Open FelipeTaiarol opened 5 years ago

FelipeTaiarol commented 5 years ago

I tried to override the offsets using the this API call but it didn't work, it looks like it ignores the offsets I sent in the request.

Also, given that I created the consumer with "auto.commit.enable": "false" I was expecting it to not commit automatically after a call to /records but it looks like it does commit the offset automatically no matter what.

Follow below how to reproduce:

create the consumer

curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" -H "Accept: application/vnd.kafka.v2+json" \
    --data '{"name": "consumer_4", "format": "json", "auto.commit.enable": "false", "auto.offset.reset": "earliest"}' \
    http://localhost:8082/consumers/test4

{"instance_id":"consumer_4","base_uri":"http://localhost:8082/consumers/test4/instances/consumer_4"}%  

subscribe to the topic:

curl -X POST -H "Content-Type: application/vnd.kafka.v2+json" \
                --data '{"topics":["test4"]}' \
    http://localhost:8082/consumers/test4/instances/consumer_4/subscription

get records

curl -X GET -H "Accept: application/vnd.kafka.json.v2+json" \
    http://localhost:8082/consumers/test4/instances/consumer_4/records

[{"topic":"test4","key":"ruum_123","value":{"action":"ADD_TASK","payload":{}},"partition":0,"offset":2}]%  

try to override offsets:

curl -X POST -H "Content-Type: application/vnd.kafka.v2+json"  \
                --data '{"offsets": [{"topic": "test4","partition": 0, "offset": 1}]}' \
 http://localhost:8082/consumers/test4/instances/consumer_4/offsets

get records again:

curl -X GET -H "Accept: application/vnd.kafka.json.v2+json" \
    http://localhost:8082/consumers/test4/instances/consumer_4/records
[]%                                                                                                                                                                              
lucasmaffazioli commented 4 years ago

I have the same problem. auto.commit = false doesn't seem to do anything.

cnwme commented 3 years ago

increase your timeout consumer and you will get records