Separator changed from . to _ when upgrading from 1.4.5 to 1.5.1

[daniele-occhipinti]

I am running Ubuntu 14.04LTS, kernel 4.4.0-109-generic.

I upgraded Telegraf from 1.4.5-1 to 1.5.1-1, and the output format changed unexpectedly, from dot-separated to underscore-separated.

This is what I was seeing on 1.4.5-1 when running tcpdump -A -i eth0 port 2003: myserver-i-00xxxxxxxx.app.io.dropwizard.jetty.MutableServletContextHandler.head-requests.count 1 1516203804

This is instead what I see on 1.5.1-1: myserver-i-00xxxxxxxx.app.io_dropwizard_jetty_MutableServletContextHandler_head-requests_count

This is my configuration, not changed during the upgrade

/etc/telegraf/telegraf.d/graphite_inputs.conf
[[inputs.udp_listener]]
data_format = "graphite"
service_address = ":8092"
templates = ["measurement.field*"]

/etc/telegraf/telegraf.d/graphite_outputs.conf
[[outputs.graphite]]
servers = ["graphite.myserver.internal:2003"]
template = "host.measurement.tags.field"
timeout = 2

[danielnelson]

Sorry about this, it looks like the problematic change was in #3473. I think this was done to improve the consistency between tag and field handling, or maybe to adjust how the field is displayed in various GUIs. I don't think there is any available workaround to revert to the prior behavior.

@PierreF It seems like we should revert this behavior, what do you think?

[daniele-occhipinti]

@danielnelson Thanks for the quick reply and for scheduling this to be fixed in 1.5.2.

A workaround we may have found is to change /etc/telegraf/telegraf.d/graphite_inputs.conf from this:

[[inputs.udp_listener]]
data_format = "graphite"
service_address = ":8092"
templates = ["measurement.field*"]

to this:

[[inputs.socket_listener]]
data_format = "graphite"
service_address = "udp://:8092"
templates = ["*.*.*.*.*.*.*.*.*.* measurement.measurement.measurement.measurement.measurement.measurement.measurement.measurement.measurement.field", 
                     "*.*.*.*.*.*.*.*.* measurement.measurement.measurement.measurement.measurement.measurement.measurement.measurement.field", 
                     "*.*.*.*.*.*.*.* measurement.measurement.measurement.measurement.measurement.measurement.measurement.field",
                     "*.*.*.*.*.*.* measurement.measurement.measurement.measurement.measurement.measurement.field",
                     "*.*.*.*.*.* measurement.measurement.measurement.measurement.measurement.field",
                     "*.*.*.*.* measurement.measurement.measurement.measurement.field", 
                     "*.*.*.* measurement.measurement.measurement.field",
                     "*.*.* measurement.measurement.field",
                     "*.* measurement.field"]

What do you reckon?

[PierreF]

@danielnelson I mitigated about if this change is wanted or a bug. The reason to qualify it as a bug seems: 1) it changed the behavior. 2) it alter the input The reason to qualify it as wanted: "." is a separator of in graphite, if we keep the "." as-is from one tag value/field name, the graphite client will see it as two elements.

I would vote for option 2 and keep the new behavior... and the other hand we missing this breaking change :(

[danielnelson]

@daniele-occhipinti Yeah I think that should work, but only when using a graphite output, with InfluxDB and other outputs though it wouldn't because the measurement and field name are combined together in the output.

@PierreF Yeah, I can see wanting it either way, especially since many graphite sources like to sprinkle in too many dots. I think our best bet is to output without the replacement, that way you can control it as needed with the separator option in the parser. This won't work if the data is from another source, but don't normally generate metrics with dots in the field name either. Do you think this is a problem, maybe we need a separator option in the serializer as well?

[PierreF]

It seems I've missed that the "." comes from graphite input. In this case the "2) it alter the input" looks much more as a bug. I would indeed expect the graphite input result (as much as possible) in the same graphite output. For myself, both behavior should works, I don't think I've ever seen any "." in fieldName (I'm only using build-in collector, no graphite input or alike).