kafka auto reset earlier did not work

[bogdan-dubyk]

As far as I know by default topic. auto.offset.reset is set to latest (correct me if I'm wrong), and with such a configuration if consumer is not listening when event dispatched, that event will be lost for a consumer, and if I understand correctly it can be changes by setting up auto.offset.reset to earliest, which i did in my local docker environment to test if it'll work.

So at the beginning I have default configuration (with offset latest) and I'm:

1. dispatching event
2. running bin/console messenger:consume {transprot_name} and as expected I did not get the message because offset is latest and at the moment of sending consumer was offline.

Now I'm want change offset ot earliest:

1. As I'm on a docker env. and to be sure offset is not set somewhere in kafka, I'm removing kafka/zookeper containers and related volumes

2. adding

     topic:
       auto.offset.reset: earliest

   to my enqueue.yaml

3. building kafka/zookeper container and run them

4. dispatching event again

5. running bin/console messenger:consume {transprot_name} afterwards again and still message is not processed by consumer

I have another project on Laravel where I did same manipulations, but except of setting auto.offset.reset to the topic section in laravel it goes to global section, and all works fine after I'm setting it earliest, consumer able to process events from the past.

What I'm doing wrong, why I can't change offset to earliest on symfony/enqueue.

Thanks a lot!

I'm using:

• PHP8
• Symfony 5.2
• enqueue/rdkafka 0.10.9

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.