search

morganstanley / modern-cpp-kafka

A C++ API for Kafka clients (i.e. KafkaProducer, KafkaConsumer, AdminClient)
Apache License 2.0
331 stars 86 forks source link

Added support for accessing cached offsets. #208

Open Fischiii opened 1 year ago

Fischiii commented 1 year ago

Added the possibility to access the local cached offsets of a consumer using rd_kafka_get_watermark_offsets as always fetching the offsets was to slow for our use case.

If there is already a way to access the currently cached offsets ignore this PR and please point me in the right direction.

kenneth-jia commented 1 year ago

Hi, @Fischiii Thank you for pointing it out! Instead of creating new interfaces (e.g. beginningOffsetsCached/endOffsetsCached), maybe we could reuse the existed ones. I'll refine the beginningOffsets(..., timeout)/endOffsets(..., timeout), to make them return cached offsets if timeout is 0. (but please wait a few more days for the PR, -- due to my access limit)

Fischiii commented 1 year ago

@kenneth-jia I applied your suggested changes. I think it is a better solution than what I first proposed.