The use of toNativeTopicPartitionListNoDispose in Kafka.Consumer.storeOffsetMessage and storeOffsets left the partition lists in memory indefinitely as there is no foreign pointer finaliser attached and the list isn't explicitly freed afterwards.
This was causing OOMs in my application every few hours.
I confirmed that using toNativeTopicPartitionList instead fixes the issue and is also in line with how these lists are handled in librdkafka (example) by being destroyed after calls to rd_kafka_offsets_store.
This PR applies the fix to storeOffsetMessage and storeOffsets.
AlexeyRaga
commented
6 months ago
The use of
toNativeTopicPartitionListNoDisposeinKafka.Consumer.storeOffsetMessageandstoreOffsetsleft the partition lists in memory indefinitely as there is no foreign pointer finaliser attached and the list isn't explicitly freed afterwards. This was causing OOMs in my application every few hours.I confirmed that using
toNativeTopicPartitionListinstead fixes the issue and is also in line with how these lists are handled inlibrdkafka(example) by being destroyed after calls tord_kafka_offsets_store.This PR applies the fix to
storeOffsetMessageandstoreOffsets.