Kafka output attempting to write to influxdb on localhost

[sjafferali]

Relevant telegraf.conf:

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 10000
  metric_buffer_limit = 50000
  collection_jitter = "5s"
  flush_interval = "10s"
  flush_jitter = "5s"
  precision = "1ns"
  debug = true
  quiet = false
  logfile = "/var/log/telegraf/telegraf.log"
  hostname = ""
  omit_hostname = false

[[outputs.kafka]]
  brokers = ['XXXXXXXX']
  topic = "MY_TOPIC"
  routing_tag = "host"

System info:

Ubuntu 16.04.2 LTS Telegraf 1.9.3-1

Steps to reproduce:

1. Add this configuration file with my Kafka topic.
2. Restart telegraf.

Expected behavior:

Debug lines should show it writing to kafka. No output should indicate writing to outputs.influxdb .

Actual behavior:

The below entries start getting logged.

2019-02-03T03:45:01Z E! [outputs.influxdb] when writing to [http://localhost:8086]: Post http://localhost:8086/write?db=telegraf: dial tcp 127.0.0.1:8086: connect: connection refused

Not sure why adding a kafka output would also make telegraf start attempting to write to influxdb. Removing the outputs.kafka config, makes this error go away.

Additional info:

Checking my kafka topic, the plugin appears to be working, just also somehow enabling outputs.influxdb plugin as well.

[danielnelson]

In the .deb package, the /etc/telegraf/telegraf.conf and all files in /etc/telegraf.d are loaded, one of them must have the influxdb output enabled (in the default config it is enabled). There must be more to the configuration since at least one input is also required. Perhaps Telegraf did not restart and is still running with the default configuration?

[sjafferali]

Hey @danielnelson . This is what I originally assumed as well, however, I can reproduce this without any other configurations apart from inputs. Additionally, if I remove the kafka.conf file, this error disappears. Attached is a tar.gz file with a full copy of the /etc/telegraf directory.

telegraf.tar.gz

The endpoints of the kafka brokers and tags have been redacted out of these files.

I also have an enterprise ticket open about this with the log file, as well as un-redacted configuration directory in ticket #40309, if you have access to this.

[danielnelson]

I think these just need to be adjusted to outputs.kafka.tagpass, otherwise they will create the parent table.

$ ack outputs.influxdb
telegraf.d/kafka.conf
99:  [outputs.influxdb.tagpass]
101:  [outputs.influxdb.tagdrop]

[sjafferali]

Excellent. Great catch. Seems that was it!

[raghav29061999]

i suppose here the issue is when you use localhost:8086 it goes to the 8086 port of docker container, while in actual the localhost you refer is your system, so you need to replace this with your ipv4 address. or in other words .conf file goes in docker container localhost within the Docker container refers to the container itself, not the hostIPV4. So we can update this to something like this

urls = [“http://:8086/”]

P.S. :smile: I know it's long since the problem occurred to you and by now you would have resolved it, but i just started in IT.