influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.89k stars 5.6k forks source link

[inputs.input_name.tags] configuration doesn't seem to do anything #2582

Closed ololduck closed 7 years ago

ololduck commented 7 years ago

Bug report

Relevant telegraf.conf:

[[inputs.logparser]]
   files = ["/var/applogs/app.log"]
   from_beginning = false
   [inputs.logparser.grok]
     patterns = ["%{APP_LOG}"]
     measurement = "metrics_logs"
     custom_patterns = '''
APP_LOG \[%{TIMESTAMP_ISO8601:logdate}\] \[%{DATA:hostname:tag}\] \[%{DATA:thread}\] %{LOGLEVEL:logLevel:tag} %{DATA:class:tag} %{DATA:method:tag} \- %{GREEDYDATA:message}
'''
  [inputs.logparser.tags]
    bname = "beatles"

[[inputs.procstat]]
  pid_file = "/var/run/myapp/myapp.pid"
  [inputs.procstat.tags]
    bname = "beatles"

System info:

root@cherrybomb ~ # uname -a
Linux cherrybomb.localdomain 4.9.13-guest #2 SMP Mon Feb 27 14:58:18 CET 2017 x86_64 x86_64 x86_64 GNU/Linux
root@cherrybomb ~ # lsb_release -a
LSB Version:    :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch
Distributor ID: CentOS
Description:    CentOS release 6.7 (Final)
Release:    6.7
Codename:   Final
root@cherrybomb ~ # telegraf --version
Telegraf v1.2.1 (git: release-1.2 3b6ffb344e5c03c1595d862282a6823ecb438cff)

Steps to reproduce:

  1. Add a tags definition to any input
  2. restart telegraf with new config file
  3. watch as no new keys are created in a influxdb output

Expected behavior:

I have a new tag in my measurement, named bname, with value beatles.

Actual behavior:

I don't have a new tag in my measurement, named bname, with value beatles.

😢

Additional info:

On reload, /var/log/telegraf/telegraf.log:

2017/03/28 16:02:16 Seeked /var/applogs/app.log - &{Offset:0 Whence:2}

I would have enjoyed to look into it myself, but i'm still strugling with go. Could someone help?

danielnelson commented 7 years ago

Are there any measurements created at all? Does it work without the tags table?

jagregory commented 7 years ago

I was just going to raise the same issue off the back of the recommended workaround to #2572.

I'm seeing measurements being created, but they don't carry any of the tags.

jagregory commented 7 years ago

After a bit of digging (bare in mind I'm unfamiliar with the codebase), I can't see where any of the inputs handle tags.

For example, logparser.go and cloudwatch.go both have the structs which are parsed from the TOML config, but neither contain a tags map anywhere. Logparser only adds tags from the log line it parses, and cloudwatch adds a few tags of it's own, but neither appear to pull any from the config.

ololduck commented 7 years ago

@danielnelson Yeah, the measurements are created, but none of them contain any tag info. Of course, the tag key isn't even sent.

danielnelson commented 7 years ago

I can't reproduce, the tag shows up. Here is what I'm doing:

[:]dbn@loaner:~/src/influx/src/github.com/influxdata/telegraf (git::(1.2.1)) $ telegraf --version
Telegraf v1.2.1 (git: HEAD 3b6ffb34)
[:]dbn@loaner:~/src/influx/src/github.com/influxdata/telegraf (git::(1.2.1)) $ cat etc/telegraf.d/input.2582.conf 
[[inputs.procstat]]
  pid_file = "/var/run/acpid.pid"
  [inputs.procstat.tags]
    bname = "beatles"
[+]dbn@loaner:~/src/influx/src/github.com/influxdata/telegraf (git::(1.2.1)) $ telegraf --config-directory etc/telegraf.d/ --test
2017/03/29 11:11:14 I! Using config file: /home/dbn/.telegraf/telegraf.conf
* Plugin: inputs.procstat, Collection 1
> procstat,pidfile=/var/run/acpid.pid,process_name=acpid,bname=beatles,host=loaner involuntary_context_switches=193i,memory_rss=1818624i,memory_swap=0i,cpu_time_nice=0,cpu_time_iowait=0,cpu_time_irq=0,memory_vms=4452352i,num_threads=1i,cpu_time_user=6.61,cpu_time_soft_irq=0,cpu_time_guest_nice=0,pid=2558i,voluntary_context_switches=340694i,cpu_time_system=31.14,cpu_time_idle=0,cpu_time_steal=0,cpu_time_stolen=0,cpu_time_guest=0 1490811074000000000
* Plugin: inputs.cpu, Collection 1
jagregory commented 7 years ago

Ok I'm bowing out of this one, my issue was I'd unintentionally defined my tags as a list [[inputs.cloudwatch.tags]]. I can see input level tags on my cloudwatch input now.

cloudwatch_aws_rds,environment=prod,...
ololduck commented 7 years ago

@danielnelson i'm closing this one.

The tags showed up. 24h late, but they showed up. I don't get it.

I don't think there is a caching system in influxdb, is there?

Sorry to have taken your time.

danielnelson commented 7 years ago

They should show up at the same time as the rest of the data. Sounds weird, if you narrow it down to a bug feel free to reopen.