search

ibm-messaging / event-streams-samples

Apache License 2.0
95 stars 168 forks source link

offset: smallest that haven't been processed? #8

Closed whatadewitt closed 8 years ago

whatadewitt commented 8 years ago

Hello,

I'm trying to leverage message hub for a project and I'm running into an issue that I am hoping someone can help with:

{ 'auto.offset.reset': ’smallest’ } -- brings back all messages for the topic in the given time period { 'auto.offset.reset': ’largest’ } -- bring back all new messages

The issue is if my consumer goes down, I want to pull back the "smallest", but only the smallest that I haven't yet processed. This seems like it should be very simple, but I'm not having much luck finding anything anywhere.

Thanks,

--d

harishvarma8055 commented 8 years ago

Hi @whatadewitt even Iam facing similar kind of issue.Can we connect regarding this issue.My mail id bravoharish69@gmail.com

whatadewitt commented 8 years ago

hi @harishvarma8055!

Try the following options when setting up your consumer...

{ 
'auto.offset.reset': 'smallest',
'auto.commit.enable': true 
}
mimaison commented 8 years ago

Hi @whatadewitt,

The auto.offset.reset policy is only used if your consumer has not committed any offsets. Otherwise, the consumer restarts from the last committed offset.

By default the consumer has auto.commit enabled, so you just need to be sure to reuse the same consumer group and you should only receive messages you've not seen already.