producer.py: increase producer timeout to 10s

astronomy-commons / adc-streaming

Core Python(ic) client libraries and utilities for alert streaming

BSD 3-Clause "New" or "Revised" License

6 stars 6 forks source link

producer.py: increase producer timeout to 10s #37

Closed myNameIsPatrick closed 4 years ago

myNameIsPatrick commented 4 years ago

See title. There were issues using the 2s timeout since that also includes the time spent to initially connect to a broker, so this has been increased to 10s. Note that the default timeout if not set is 300s.

Ideally, there would be two separate timeouts:

10s timeout to connect to a broker
- while connected, 2s timeout to send a message

but I didn't see anything more fine-grained in looking at the various librdkafka settings. Could've have easily missed the magic setting though.

spenczar commented 4 years ago

For posterity - @myNameIsPatrick had logs that showed around 100ms RTT talking to the dev.hop.scimma.org brokers. Between opening a TCP connection, fetching a list of broker addresses from the bootstrap broker, opening a TLS connection, and doing an SSL handshake, there are a lot of roundtrips when first connecting to Kafka, adding up to around 2.0s-2.5s typically. But the Producer's timeout clock starts immediately, before the connection is established, so it was common for the first message to timeout without this change.