How to escape offline messages to Consumer?

[Ryan-Lew]

Consumer subscribes topics. Consumer recieve many offline messages when reconnect to Kafka server.How to escape the offline messages?I have tested by setting topic config "auto.offset.reset" to "latest" and "smallest", but no effect.

[mfontanini]

Are you committing any offsets when you receive messages? That's the way to say "I acknowledge this message (and every other before it in this partition) and don't want to see it again".

[accelerated]

@Ryan-Lew, you can set auto.offset.reset=latest and also when you assign your partitions on startup by calling assign() (either in the rebalance callback if you use dynamic partition assignment via subscribe) or by calling it directly (static assignment), set the starting offset to TopicPartition::OFFSET_END for all your partitions. Also you must commit each message afterwards as indicated above.

[Ryan-Lew]

@Ryan-Lew, you can set auto.offset.reset=latest and also when you assign your partitions on startup by calling assign() (either in the rebalance callback if you use dynamic partition assignment via subscribe) or by calling it directly (static assignment), set the starting offset to TopicPartition::OFFSET_END for all your partitions. Also you must commit each message afterwards as indicated above.

Thanks! So, I have to assign my partitions by calling 'assign()' not 'subscribe()'?

[accelerated]

Subscribe doesn't have offsets AFIK. So yes, just using assign.

[Ryan-Lew]

ok,thank you.