multiple sections - but only first rendered

[ZyanKLee]

Hey, from @nrdmn 's suggestions at https://github.com/yankcrime/puppet-telegraf/issues/96#issuecomment-363081152 I modified the hieradata, manifests and profiles to match 2.0 syntax. This works now, but I do encounter an unwanted behaviour nevertheless. It seems like only the first section is rendered at all (even with some more patterns, nginx is the only one that gets written to telegraf.conf, when I change the sequence then the new first one is going to be written to conf instead):

What I did:

telegraf::inputs:
  procstat:
    - pattern: "nginx"
      pid_tag: true
    - pattern: "php.*-fpm"
      pid_tag: true

What I expected:

[[inputs.procstat]]
  pattern = "nginx"
  pid_tag = true

[[inputs.procstat]]
  pattern = "php.*-fpm"
  pid_tag = true

What I got instead:

[[inputs.procstat]]
pattern = "nginx"
pid_tag = true

Can this be a result of using the deep_merge gem?

System Specs

• ubuntu 16.04 (all systems)
• puppet agent 4.10.9
• puppetserver version: 2.8.0
• yankcrime/telegraf 2.0.0

[ZyanKLee]

@yankcrime @nrdmn any idea? this problem still persists and documentation in README.md does not help with it.

[ZyanKLee]

more information on this matter:

• hiera v5
• gems deep_merge, toml-rb are installed

• 99 does apparently not resolve my problem

• output of puppet lookup --environment production --merge deep --render-as yaml --node nodename "telegraf::inputs" is correct
• this issue is not limited to procstat input (tested with nginx, too)
• there is no spec test for this situation
• README.md is outdated

[ZyanKLee]

full (deep merged) example (output of above lookup command):

hiera:

---
nginx:
- urls:
  - http://localhost/nginx_status
  response_timeout: 5s
- urls:
  - http://127.0.0.1/nginx_status
  response_timeout: 5s
procstat:
- pattern: nginx
  pid_tag: true
- pattern: php.*-fpm
  pid_tag: true
cpu:
- percpu: true
  totalcpu: true
disk:
- ignore_fs:
  - tmpfs
  - devtmpfs
  - devfs
diskio:
- {}
kernel:
- {}
mem:
- {}
processes:
- {}
swap:
- {}
system:
- {}
net:
- interfaces:
  - eth*
  - en*
netstat:
- {}

telegraf.conf

#
# INPUTS:
#
[[inputs.cpu]]
percpu = true
totalcpu = true

[[inputs.disk]]
ignore_fs = ["tmpfs", "devtmpfs", "devfs"]

[[inputs.diskio]]

[[inputs.kernel]]

[[inputs.mem]]

[[inputs.net]]
interfaces = ["eth*", "en*"]

[[inputs.netstat]]

[[inputs.nginx]]
response_timeout = "5s"
urls = ["http://localhost/nginx_status"]

[[inputs.processes]]

[[inputs.procstat]]
pattern = "nginx"
pid_tag = true

[[inputs.swap]]

[[inputs.system]]

[corentingi]

I've just stumbled upon the same problem. The module configuration basically doesn't follow the "object list" pattern that double square braquets actually mean... The only solution I found is to just create separate files.

[ragonlan]

Same problem here but with windows. Multiples 'win_perf_counters.object' but only last is in telegraf configuration file.

[ZyanKLee]

I lost track of this project - is this still a valid issue? Else I would suggest to close this ticket.

[linuxmail]

Hi,

I have same problem, twice ... first ... I was unable to add two snmp inputs, for different hosts and options (sha/public ...), but I reconfigured the hosts, to avoid the problem. Now I have something like:

## Realtime instance
[[inputs.vsphere]]
## List of vCenter URLs to be monitored. These three lines must be uncommented
## and edited for the plugin to work.
interval = "60s"
  vcenters = [ "https://192.168.1.1/sdk" ]
  username = "foo"
  password = "bar"

vm_metric_include = ["*"]
host_metric_include = []
cluster_metric_include = []
#datastore_metric_exclude = ["*"]

max_query_metrics = 256
timeout = "60s"
insecure_skip_verify = true

## Historical instance
[[inputs.vsphere]]
interval = "300s"
  vcenters = [ "https://192.168.1.1/sdk" ]
  username = "foo"
  password = "bar"

  datastore_metric_include = [ "disk.capacity.latest", "disk.used.latest", "disk.provisioned.latest"]
  insecure_skip_verify = true
  force_discover_on_init = true
  host_metric_exclude = ["*"] # Exclude realtime metrics
  vm_metric_exclude = ["*"] # Exclude realtime metrics

  max_query_metrics = 256
  collect_concurrency = 3

So I have no idea, how to get it working with Hiera. Maybe separate filenames are a solution, but I have problems with the syntax .. How to specify create files, instead of /etc/telegraf/telegraf.conf with Hiera ?

cu denny

Update

I've got it working ... had an old version from the module ... as example the vsphere plugin:

telegraf::inputs:
  vsphere:
    - vcenters:
       - "https://192.168.1.1/sdk"
      plugin_type: "vsphere"
      interval: '60s'
      username: "icinga@example.local"
      password: "%{hiera('monitoring::vmware::dc::user::password')}"
      vm_metric_include: "['*']"
      host_metric_include: '[]'
      cluster_metric_include: '[]'
      max_query_metrics: 256
      timeout: '60s'
      insecure_skip_verify: true

    - vcenters:
       - "https://192.168.1.1/sdk"
      plugin_type: "vsphere"
      interval: '300s'
      username: "icinga@example.local"
      password: "%{hiera('monitoring::vmware::dc::user::password')}"
      vm_metric_include: "['*']"
      datastore_metric_include: [ "disk.capacity.latest", "disk.used.latest", "disk.provisioned.latest"] 
      max_query_metrics: 256
      timeout: '60s'
      collect_concurrency: 3
      insecure_skip_verify: true

So now both are rendered correctly. In the same way, SNMP should work too :-)

[trunet]

toml-rb doesn't indent correctly and restarting the service causes:

Jun 25 14:43:44 [REDACTED] telegraf[27450]: 2020-06-25T13:43:44Z E! [telegraf] Error running agent: Error parsing /etc/telegraf/telegraf.conf, line 37: invalid TOML syntax

line 37 is percpu = true. note that there's no indentation.

#
# INPUTS:
#
[[inputs.cpu]]
percpu = true
totalcpu = true
[[inputs.disk]]
[[inputs.diskio]]
[[inputs.kernel]]
[[inputs.mem]]
[[inputs.net]]
[[inputs.ntpq]]
[[inputs.processes]]
[[inputs.swap]]
[[inputs.system]]