search

narcisoguillen / kafka-node-avro

ISC License
26 stars 13 forks source link

fromOffset:'earliest' #1

Closed pscanlon1 closed 5 years ago

pscanlon1 commented 5 years ago

Hey thanks for the great library... I am trying to consume from offset 0 to no avail.. im doing something like

let consumer = kafka.addConsumer("tx_table",{fromOffset:'latest'});

its not starting immediately, but if I start producing messages I get results... any ideas?

pscanlon1 commented 5 years ago

i was able to workaround this by doing the following

let consumer = kafka.addConsumer("tx_table",{fromOffset:'earliest', groupId: new Date().getTime().toString()});

narcisoguillen commented 5 years ago

Hey thanks for using the library, and thanks for helping making it better! . I'm planning on making the groupId a required field. or at least set it by default as you mention but like 

Date.now().toString()
narcisoguillen commented 5 years ago

I took a look into kafka-node library but did not figured how to send the --from-beginning and since kafka-node sets by default the groupId as 'kafka-node-group' when you create the consumer and its on the same groupId, the consumer is not getting the messages form offset 0. your workaround is a good way to accomplish this.

narcisoguillen commented 5 years ago

I will continue to close this issue since its directly related to how kafka-node interacts with the topic messages though the ConsumerGroup.