search

dpkp / kafka-python

Python client for Apache Kafka
http://kafka-python.readthedocs.io/
Apache License 2.0
5.58k stars 1.4k forks source link

delivery_timeout_ms for KafkaProducer (KIP-91) #1723

Open andriika opened 5 years ago

andriika commented 5 years ago

delivery_timeout_ms is missing in producer default config: https://github.com/dpkp/kafka-python/blob/master/kafka/producer/kafka.py#L280

I want to be able to overwrite delivery.timeout.ms in producer config

dpkp commented 5 years ago

This feature (KIP-91) has not been implemented for kafka-python. Happy to take PRs!

isamaru commented 5 years ago

This would be greatly appreciated! We intended to use low request_timeout_ms for fast retry of requests, but we are running into the issue described in KIP-91: batches are being expired without retrying, causing message loss!

As a workaround, we have to use large timeouts and can't leverage retries.