Currently any Kafka related messages are logged to the standard dev channel. Kafka consumers constantly loose and regain connections and whenever this happens, log messages are logged. This becomes rather burdensome when troubleshooting issues (especially unrelated to kafka) as the logs are spammed with these messages. This causes the logs to rotate more often than necessary in addition to making the logs larger and more difficult to sift through while troubleshooting unrelated issues to kafka.
It would be ideal if we could log Kafka related messages to its own log channel, much like how pebble and kv have their own logs.
In summary, the ask is:
Please create a separate log for Kafka messages, just like the separate logs for kv and pebble, and remove these messages from the dev channel.
blathers-crl[bot]
commented
3 weeks ago
Currently any Kafka related messages are logged to the standard dev channel. Kafka consumers constantly loose and regain connections and whenever this happens, log messages are logged. This becomes rather burdensome when troubleshooting issues (especially unrelated to kafka) as the logs are spammed with these messages. This causes the logs to rotate more often than necessary in addition to making the logs larger and more difficult to sift through while troubleshooting unrelated issues to kafka.
It would be ideal if we could log Kafka related messages to its own log channel, much like how pebble and kv have their own logs.
In summary, the ask is:
Please create a separate log for Kafka messages, just like the separate logs for kv and pebble, and remove these messages from the dev channel.
Jira issue: CRDB-41670
Jira issue with more context: CRDB-39717