search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.7k stars 5.59k forks source link

Cancel gracefully when connecting to kafka broker #12214

Open reimda opened 2 years ago

reimda commented 2 years ago

If the user sends a signal to telegraf while kafka_consumer is connecting to the broker (before sarama.NewConsumerGroup() has returned), the signal will be delayed until connection is made or timed out. This means sigint to shut down telegraf, or sighup to reload config are ignored while connecting to the broker.

Since #12111, telegraf allows the kafka connection to be deferred until after startup and to retry for a very long time in case the broker is down. This means the time spent connecting to the broker can be much larger and signals can be blocked indefinitely.

It doesn't look like there is a way to cancel sarama.NewConsumerGroup(). We need to find out if there is a different way to connect that has a way to cancel, or work with upstream to add one.

powersj commented 1 year ago

next steps: investigate what can be done to cancel the consumer group, or look to document the issue