search

grafana / carbon-relay-ng

Fast carbon relay+aggregator with admin interfaces for making changes online - production ready
Other
467 stars 151 forks source link

Kafka output needs improved error logging #186

Closed tehlers320 closed 7 years ago

tehlers320 commented 7 years ago

Setup kafka using a line like this: 'addRoute kafkaMdm mdm kafka.monitoring.foapp.com:9092 mdm none /etc/carbon-relay-ng/storage-schemas.conf bySeries 1',

start relay 1 no error ouput start relay 2 

18:35:49.677733 ▶ NOTI  kafkaMdm "mdm": now connected to kafka
18:35:51.798208 ▶ WARN  KafkaMdm "mdm": failed to submit data: kafka: Failed to deliver 56 messages. will try again in 100ms. (this attempt took 1.145096886s)
18:35:52.921394 ▶ WARN  KafkaMdm "mdm": failed to submit data: kafka: Failed to deliver 56 messages. will try again in 100ms. (this attempt took 1.022983538s)

I am not sure why.

I have kafka 0.10.2.1 in this setup and 64 partitions.

tehlers320 commented 7 years ago

Well this is embarrassing. It seems instance 1 had a different resolver than instance 2. I am unsure how any of this changed. Anyhow, i think this highlights the need for better logging.

I will try and add some on my free time and submit a PR. I think i know how to get more details from sarama after reading the docs.

Changing the title on this bug.