dpkp / kafka-python

Python client for Apache Kafka
http://kafka-python.readthedocs.io/
Apache License 2.0
5.59k stars 1.4k forks source link

KafkaProducer cannot send data standalone #2287

Open jason19970210 opened 2 years ago

jason19970210 commented 2 years ago

Try to compare these 2 sample below

if __name__ == '__main__':
    producer.send(topic, value=b'message')
if __name__ == '__main__':
    while 1:
        producer.send(topic, value=b'message')
        time.sleep(2)

So once standalone the producer.send, the consumer cant get the new data, but if we add the while 1 statement, then messages are run regularly.

Please have a look on the way to explain or fix !!! Thanks a lot !!!

vish0l commented 2 years ago

@jason19970210 use .flush() on producer at the end.

if __name__ == '__main__':
    producer.send(topic, value=b'message')
    producer.flush()

for content KafkaProducer send message in async call, first add message in local buffer and than send it to topic. in first case your script will exit with KafkaProducer instance immediately after adding it to local buffer even before producer get change to send it to topic.