inputs.disk skipped sdb drive

Badb0yBadb0y commented 3 weeks ago

Relevant telegraf.conf

[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "overlay", "aufs", "squashfs"]

Logs from Telegraf

This can see the sdb disk:
sudo -u infra-telegraf cat /proc/self/mounts | grep sdb
/dev/sdb1 /data/kafka xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=128k,sunit=256,swidth=512,noquota 0 0

This showing nothing only sda related:
grep 'disk,device=sdb' /tmp/metrics.out 

debug log can see sdb:
infra-telegraf --debug --config /etc/infra-telegraf/infra-telegraf.d/infra-base.conf --test |grep sdb|grep disk|grep -v diskio
2024-10-30T07:55:45Z I! Loading config: /etc/infra-telegraf/infra-telegraf.d/infra-base.conf
2024-10-30T07:55:45Z I! Starting Telegraf 1.32.2 brought to you by InfluxData the makers of InfluxDB
2024-10-30T07:55:45Z I! Available plugins: 235 inputs, 9 aggregators, 32 processors, 26 parsers, 62 outputs, 6 secret-stores
2024-10-30T07:55:45Z I! Loaded inputs: cpu disk diskio exec kernel mem net netstat processes smart swap system temp
2024-10-30T07:55:45Z I! Loaded aggregators:
2024-10-30T07:55:45Z I! Loaded processors:
2024-10-30T07:55:45Z I! Loaded secretstores:
2024-10-30T07:55:45Z W! Outputs are not used in testing mode!
2024-10-30T07:55:45Z I! Tags enabled: host=sfdfsfsf
2024-10-30T07:55:45Z D! [agent] Initializing plugins
2024-10-30T07:55:45Z W! DeprecationWarning: Value "false" for option "ignore_protocol_stats" of plugin "inputs.net" deprecated since version 1.27.3 and will be removed in 1.36.0: use the 'inputs.nstat' plugin instead for protocol stats
2024-10-30T07:55:45Z D! [agent] Starting service inputs
2024-10-30T07:55:46Z D! [agent] Stopping service inputs
2024-10-30T07:55:46Z D! [agent] Input channel closed
2024-10-30T07:55:46Z D! [agent] Stopped Successfully
> disk,device=sdb1,fstype=xfs,host=dsfdsfsdfdsf,mode=rw,path=/data/kafka free=25641839218688u,inodes_free=2916264436u,inodes_total=2916292608u,inodes_used=28172u,inodes_used_percent=0.0009660210337850982,total=29860699308032u,used=4218860089344u,used_percent=14.128470488329123 1730274946000000000

In the metrics output which is actually would sent it over no sdb.

System info

ubuntu 22.04 with telegraf 1.24.3 or telegraf 1.32.1

Docker

No response

Steps to reproduce

Just use the above config part and it will not be reported the disk, only diskio for that specific sdb drive.

Expected behavior

sdb drive should be shown as this command can see it: sudo -u telegraf cat /proc/self/mounts | grep sdb

Actual behavior

sdb totally skipped from metrics

Additional info

No response

srebhan commented 3 weeks ago

Well I see

> disk,device=sdb1,fstype=xfs,host=dsfdsfsdfdsf,mode=rw,path=/data/kafka free=25641839218688u,inodes_free=2916264436u,inodes_total=2916292608u,inodes_used=28172u,inodes_used_percent=0.0009660210337850982,total=29860699308032u,used=4218860089344u,used_percent=14.128470488329123 1730274946000000000

in the output (second last line) so I wonder what you are asking about?

Badb0yBadb0y commented 3 weeks ago

in the output (second last line) so I wonder what you are asking about?

Found the issue, so in the disk troubleshooting steps these are written:

$ sudo -u telegraf cat /proc/self/mounts | grep sda2
/dev/sda2 /home ext4 rw,relatime,data=ordered 0 0
$ sudo -u telegraf stat /home

The 2nd condition (stat) is actually AND not OR and there we didn't have proper directory permission, telegraf couldn't read the directory under the main folder, only the main folder.

It works now.

influxdata / telegraf