Telegraf update broke [[inputs.disk]] functionality

Posting here because of my post over at the forums.

Bug report

Running Telegraf, InfluxDB and Grafana on unRAID the [[inputs.disk]] plug-in does not seem to work. Grafana can not find any data to display, but no error messages come up in either programs logs. Everything worked fine until 6am UTC+1 on the 14th of November, but after my docker containers automatically updated everything stopped working. If it is needed, I can provide a link to connect to my InfluxDB database but I'd rather just share that as a private message.

Relevant telegraf.conf & logs:

config on Pastebin Telegraf log InfluxDB log Grafana log

System info:

unRAID 6.3.5 (Linux based)
running the official docker containers of Telegraf, InfluxDB and Grafana
all containers updated to the latest versions (Telegraf 1.4.4, InfluxDB 1.4.2, Grafana 4.2)
all containers running on the same machine
Pentium G4600, 16GB RAM, multiple HDDs

Steps to reproduce:

Install the above mentioned containers
Map paths for all containers this way (99.9% sure it is how it's supposed to be)
configure Telegraf to output data to InfluxDB (config file above)
configure [[inputs.disk]] with default settings, monitoring all mount points
add database "telegraf" to Grafana
set graph up like this to show HDD space

Expected behavior:

Telegraf should be feeding metrics for the disks into InfluxDB and Grafana should display these in its graphs.

Actual behavior:

Apparently there are no metrics gathered for the disks although the [[inputs.disk]] plugin is enabled. Grafana states "No data points" on graphs and "No data to show" in tables.

Thanks for opening such a detailed report. Can you collect the contents of /rootfs/proc/self/mounts and /rootfs/proc/filesystems in the container, also if possible can you try rolling back to 1.4.3 and see if the disk input begins working again?

@danielnelson What exactly do you mean by "collecting the contents"? Sorry, I'm fairly new to this. I know how to enter bash-enabled docker containers but I guess the Telegraf container does not have bash included. I'm always receiving "rpc error: code = 2 desc = containerd: container not started" which prevents me from ssh'ing into it.

Also, I don't actually know how I'd roll back to an older version of the container tbh. Never found a version history or anything on docker hub.

I'm just referring to reading the file with cat /rootfs/proc/self/mounts or such. Each docker image is tagged with the version number, so you can, for example, docker pull telegraf:1.4.3 to get an older version.

Rolling back to 1.4.3 actually did the trick. Everything's working like a charm now!

But still, this is bugging me. I'd really like to know what's causing it so feel free to squeeze every bit of information out of me that you need to investigate this.

I figured out how to add bash to the container so

for Telegraf version 1.4.4 here's /rootfs/proc/self/mounts and there's /rootfs/proc/filesystems
for Telegraf version 1.4.3 here's /rootfs/proc/self/mounts and there's rootfs/proc/filesystems.

For the mounts files, there's a change in lines 118 and 119 that might be interesting. The filesystems files are identical.

Just ran across this after having the same issue and I can confirm the same behavior as @sebastianklein96 on 1.4.4-alpine. The only action I took was to roll back to 1.4.3-alpine and it seems to have solved the issue for me.

@ccmpbll to get this working, remove the HOST_MOUNT_PREFIX environment variable from your docker config and use /rootfs/mnt/**PATHtoDISK** instead of /mnt/**PATHtoDISK

Or take a look at this pull.

@sebastianklein96 Yep, I read both of those before posting, but I am curious about something. Would removing that environment variable not cause a new series to be created in the InfluxDB? I didn't want to lose my historical data in the graphs, so I didn't try it.

@ccmpbll yeah, I guess that's the downside to this method. But hey, atleast it works 😉

Might be best to stick with 1.4.3 for now, and I'll try to get this fixed for 1.4.5, pending some research about how the various environment variables affect the output.

I'm looking at this again but my 1.4.4 and 1.4.3 seem to act identically so it would seem I am not triggering the bug:

1.4.3 with HOST_MOUNT_PREFIX

[+]dbn@debian-stretch-docker:~$ docker run --rm --name telegraf --privileged -v $(pwd)/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc -e HOST_MOUNT_PREFIX=/rootfs telegraf:1.4.3 /usr/bin/telegraf --test
2017/11/29 21:55:56 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.disk, Collection 1
> disk,fstype=overlay,host=6c50c16367df,path=/,device=overlay free=1887207424i,used=4123721728i,used_percent=68.60373203081134,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i 1511992556000000000
> disk,path=/etc/resolv.conf,device=vda1,fstype=ext4,host=6c50c16367df total=6306725888i,free=1887207424i,used=4123721728i,used_percent=68.60373203081134,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992556000000000
> disk,path=/etc/hostname,device=vda1,fstype=ext4,host=6c50c16367df total=6306725888i,free=1887207424i,used=4123721728i,used_percent=68.60373203081134,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992556000000000
> disk,path=/etc/hosts,device=vda1,fstype=ext4,host=6c50c16367df used_percent=68.60373203081134,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887207424i,used=4123721728i 1511992556000000000
> disk,device=vda1,fstype=ext4,host=6c50c16367df,path=/etc/telegraf/telegraf.conf inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887207424i,used=4123721728i,used_percent=68.60373203081134 1511992556000000000

1.4.4 with HOST_MOUNT_PREFIX

[+]dbn@debian-stretch-docker:~$ docker run --rm --name telegraf --privileged -v $(pwd)/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc -e HOST_MOUNT_PREFIX=/rootfs telegraf:1.4.4 /usr/bin/telegraf --test
2017/11/29 21:55:59 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.disk, Collection 1
> disk,path=/,device=overlay,fstype=overlay,host=1dff71896a82 inodes_used=111081i,total=6306725888i,free=1887186944i,used=4123742208i,used_percent=68.60407274352782,inodes_total=393216i,inodes_free=282135i 1511992560000000000
> disk,path=/etc/resolv.conf,device=vda1,fstype=ext4,host=1dff71896a82 inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887186944i,used=4123742208i,used_percent=68.60407274352782 1511992560000000000
> disk,path=/etc/hostname,device=vda1,fstype=ext4,host=1dff71896a82 total=6306725888i,free=1887186944i,used=4123742208i,used_percent=68.60407274352782,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992560000000000
> disk,path=/etc/hosts,device=vda1,fstype=ext4,host=1dff71896a82 inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887186944i,used=4123742208i,used_percent=68.60407274352782 1511992560000000000
> disk,device=vda1,fstype=ext4,host=1dff71896a82,path=/etc/telegraf/telegraf.conf inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887186944i,used=4123742208i,used_percent=68.60407274352782 1511992560000000000

I do see more devices without HOST_MOUNT_PREFIX, but again the output is the same between the two versions:

1.4.3 without HOST_MOUNT_PREFIX

[+]dbn@debian-stretch-docker:~$ docker run --rm --name telegraf --privileged -v $(pwd)/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc telegraf:1.4.3 /usr/bin/telegraf --test
2017/11/29 21:57:56 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.disk, Collection 1
> disk,host=d3160156d6ca,path=/,device=overlay,fstype=overlay inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i 1511992676000000000
> disk,path=/rootfs,device=vda1,fstype=ext4,host=d3160156d6ca total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992676000000000
> disk,path=/rootfs/var/lib/docker/plugins,device=vda1,fstype=ext4,host=d3160156d6ca inodes_used=111081i,total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i,inodes_free=282135i 1511992676000000000
> disk,path=/rootfs/var/lib/docker/overlay2,device=vda1,fstype=ext4,host=d3160156d6ca total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992676000000000
> disk,host=d3160156d6ca,path=/rootfs/var/lib/docker/overlay2/f1c22781751cbdf49088b94bcaa2fb030baa3ce0b26f52e2de75e96d451d2ea8/merged,device=overlay,fstype=overlay used_percent=68.60679844525973,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887023104i,used=4123906048i 1511992676000000000
> disk,path=/rootfs/var/lib/docker/overlay2/f1c22781751cbdf49088b94bcaa2fb030baa3ce0b26f52e2de75e96d451d2ea8/merged,device=overlay,fstype=overlay,host=d3160156d6ca inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i 1511992676000000000
> disk,device=vda1,fstype=ext4,host=d3160156d6ca,path=/rootfs/mnt/home inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973 1511992676000000000
> disk,fstype=ext4,host=d3160156d6ca,path=/rootfs/etc,device=vda1 total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992676000000000
> disk,path=/etc/resolv.conf,device=vda1,fstype=ext4,host=d3160156d6ca total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992676000000000
> disk,path=/etc/hostname,device=vda1,fstype=ext4,host=d3160156d6ca total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992676000000000
> disk,device=vda1,fstype=ext4,host=d3160156d6ca,path=/etc/hosts inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973,inodes_total=393216i 1511992676000000000
> disk,host=d3160156d6ca,path=/etc/telegraf/telegraf.conf,device=vda1,fstype=ext4 inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887023104i,used=4123906048i,used_percent=68.60679844525973 1511992676000000000

1.4.4 without HOST_MOUNT_PREFIX

[+]dbn@debian-stretch-docker:~$ docker run --rm --name telegraf --privileged -v $(pwd)/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc telegraf:1.4.4 /usr/bin/telegraf --test
2017/11/29 21:58:00 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.disk, Collection 1
> disk,fstype=overlay,host=289929ea5c07,path=/,device=overlay total=6306725888i,free=1887010816i,used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992681000000000
> disk,path=/rootfs,device=vda1,fstype=ext4,host=289929ea5c07 inodes_used=111081i,total=6306725888i,free=1887010816i,used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i 1511992681000000000
> disk,path=/rootfs/var/lib/docker/plugins,device=vda1,fstype=ext4,host=289929ea5c07 inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887010816i,used=4123918336i,used_percent=68.60700287288962 1511992681000000000
> disk,path=/rootfs/var/lib/docker/overlay2,device=vda1,fstype=ext4,host=289929ea5c07 total=6306725888i,free=1887010816i,used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992681000000000
> disk,device=overlay,fstype=overlay,host=289929ea5c07,path=/rootfs/var/lib/docker/overlay2/aebeb9226349585981b8aa89dd36c4e045ad4a802b1261618197c3f4ae7b3797/merged used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887010816i,used=4123918336i 1511992681000000000
> disk,path=/rootfs/var/lib/docker/overlay2/aebeb9226349585981b8aa89dd36c4e045ad4a802b1261618197c3f4ae7b3797/merged,device=overlay,fstype=overlay,host=289929ea5c07 total=6306725888i,free=1887010816i,used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i 1511992681000000000
> disk,path=/rootfs/mnt/home,device=vda1,fstype=ext4,host=289929ea5c07 inodes_used=111081i,total=6306725888i,free=1887010816i,used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i 1511992681000000000
> disk,host=289929ea5c07,path=/rootfs/etc,device=vda1,fstype=ext4 used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887010816i 1511992681000000000
> disk,path=/etc/resolv.conf,device=vda1,fstype=ext4,host=289929ea5c07 free=1887010816i,used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i 1511992681000000000
> disk,path=/etc/hostname,device=vda1,fstype=ext4,host=289929ea5c07 free=1887010816i,used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i 1511992681000000000
> disk,fstype=ext4,host=289929ea5c07,path=/etc/hosts,device=vda1 inodes_used=111081i,total=6306725888i,free=1887010816i,used=4123918336i,used_percent=68.60700287288962,inodes_total=393216i,inodes_free=282135i 1511992681000000000
> disk,path=/etc/telegraf/telegraf.conf,device=vda1,fstype=ext4,host=289929ea5c07 inodes_total=393216i,inodes_free=282135i,inodes_used=111081i,total=6306725888i,free=1887010816i,used=4123918336i,used_percent=68.60700287288962 1511992681000000000

I don't have unRAID, not sure if this is important. @ccmpbll are you also using unRAID?

@sebastianklein96 @ccmpbll I know you normally have a GUI to setup the containers, but is it possible to run commands like above to check if they produce the change?

Hey @danielnelson, nice seeing you look into this further!

I'm happy to run the commands but to be honest I'm not that knowledgeable when it comes to docker commands. Could you maybe check if I got the use of the single command parts right?

If we take the first example, 1.4.3 with HOST_MOUNT_PREFIX, I've got this so far:

Command snippet	Used for
docker run --rm --name telegraf	run Telegraf and remove when exiting
--privileged	run privileged
-v $(pwd)/telegraf.conf:/etc/telegraf/telegraf.conf:ro	telegraf.conf location - What's $ (pwd)/ ?
-v /var/run/utmp:/var/run/utmp:ro	mount utmp folder
-v /:/rootfs:ro	mount root to rootfs
-v /sys:/rootfs/sys:ro	mount sys to rootfs
-v /etc:/rootfs/etc:ro	mount etc to rootfs
-v /proc:/rootfs/proc:ro	mount proc to rootfs
-v /var/run/docker.sock:/var/run/docker.sock	added this to connect to docker
-e HOST_PROC=/rootfs/proc	set HOST_PROC variable
-e HOST_SYS=/rootfs/sys	set HOST_SYS variable
-e HOST_ETC=/rootfs/etc	set HOST_ETC variable
-e HOST_MOUNT_PREFIX=/rootfs	set HOST_MOUNT_PREFIX location
telegraf:1.4.3 /usr/bin/telegraf	use telegraf 1.4.3 in folder /usr/bin/telegraf
--test	what does this do?

So, what does $(pwd)/telegraf.conf do? I figured out that it sets the path of the config file on the host system, but what does your abbreviation of the path do? And what is --test for? Is it just for verbose output?

If I assume that I've got everything right and I've adapted it correctly, I'm getting these outputs:

Telegraf 1.4.3 with HOST_MOUNT_PREFIX

Docker command: docker run --rm --name telegraf143test --privileged -v /mnt/user/appdata/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc -e HOST_MOUNT_PREFIX=/rootfs telegraf:1.4.3 /usr/bin/telegraf --test

Output

2017/11/30 03:21:40 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.disk, Collection 1
> disk,fstype=vfat,host=bfc2ca632aeb,path=/boot,device=sdb1 inodes_used=0i,total=15359926272i,free=15149989888i,used=209936384i,used_percent=1.3667798938768239,inodes_total=0i,inodes_free=0i 1512012101000000000
> disk,path=/mnt,device=/mnt,fstype=none,host=bfc2ca632aeb inodes_free=2009602i,inodes_used=20931i,total=8317063168i,free=7523766272i,used=793296896i,used_percent=9.538185294206004,inodes_total=2030533i 1512012101000000000
> disk,host=bfc2ca632aeb,path=/mnt/disk1,device=md1,fstype=xfs used_percent=62.54008126792389,inodes_total=293026624i,inodes_free=292669784i,inodes_used=356840i,total=2999127797760i,free=1123470835712i,used=1875656962048i 1512012101000000000
> disk,path=/mnt/disk2,device=md2,fstype=xfs,host=bfc2ca632aeb total=2999127797760i,free=1494033113088i,used=1505094684672i,used_percent=50.18441314158506,inodes_total=293026624i,inodes_free=293014737i,inodes_used=11887i 1512012101000000000
> disk,path=/mnt/disk3,device=md3,fstype=xfs,host=bfc2ca632aeb total=2999127797760i,free=1494415253504i,used=1504712544256i,used_percent=50.17167142326664,inodes_total=293026624i,inodes_free=293021669i,inodes_used=4955i 1512012101000000000
> disk,path=/mnt/disk4,device=md4,fstype=xfs,host=bfc2ca632aeb total=2999127797760i,free=1498240638976i,used=1500887158784i,used_percent=50.04412149108779,inodes_total=293026624i,inodes_free=293013129i,inodes_used=13495i 1512012101000000000
> disk,path=/mnt/disk5,device=md5,fstype=xfs,host=bfc2ca632aeb total=2999127797760i,free=1497535639552i,used=1501592158208i,used_percent=50.06762830612003,inodes_total=293026624i,inodes_free=293005790i,inodes_used=20834i 1512012101000000000
> disk,device=sdi1,fstype=btrfs,host=bfc2ca632aeb,path=/mnt/cache inodes_total=0i,inodes_free=0i,inodes_used=0i,total=128035643392i,free=81507979264i,used=45542633472i,used_percent=35.84605574995029 1512012101000000000
> disk,path=/mnt/user0,device=shfs,fstype=fuse.shfs,host=bfc2ca632aeb inodes_total=1465133120i,inodes_free=1464725109i,inodes_used=408011i,total=14995638988800i,free=7107695480832i,used=7887943507968i,used_percent=52.601583125996676 1512012101000000000
> disk,path=/mnt/user,device=shfs,fstype=fuse.shfs,host=bfc2ca632aeb total=15123674632192i,free=7189203460096i,used=7933486141440i,used_percent=52.46081451433218,inodes_total=1465133120i,inodes_free=1464725109i,inodes_used=408011i 1512012101000000000
> disk,device=loop0,fstype=btrfs,host=bfc2ca632aeb,path=/etc/libvirt inodes_free=0i,inodes_used=0i,total=1073741824i,free=948502528i,used=17170432i,used_percent=1.7780794027824909,inodes_total=0i 1512012101000000000
> disk,path=/var/lib/docker,device=loop1,fstype=btrfs,host=bfc2ca632aeb total=42949672960i,free=31778615296i,used=8732020736i,used_percent=21.554884324952187,inodes_total=0i,inodes_free=0i,inodes_used=0i 1512012101000000000
> disk,path=/mnt/disks/2TB,device=sda1,fstype=fuseblk,host=bfc2ca632aeb free=1727088148480i,used=273273991168i,used_percent=13.661225922626565,inodes_total=1686707824i,inodes_free=1686695079i,inodes_used=12745i,total=2000362139648i 1512012101000000000
* Plugin: inputs.diskio, Collection 1
> diskio,host=bfc2ca632aeb,name=sdi reads=7794237i,read_bytes=1971918573568i,write_bytes=3918547959808i,write_time=1611059925i,weighted_io_time=1685953902i,writes=12210046i,read_time=74891165i,io_time=30089913i,iops_in_progress=0i 1512012101000000000
> diskio,name=md5,host=bfc2ca632aeb writes=96628i,read_bytes=1848537799168i,write_time=0i,reads=14146328i,write_bytes=87622718464i,read_time=0i,io_time=0i,weighted_io_time=0i,iops_in_progress=0i 1512012101000000000
> diskio,host=bfc2ca632aeb,name=sde reads=24597510i,read_bytes=5170723778048i,write_time=11620349i,iops_in_progress=0i,writes=378468i,write_bytes=224203014144i,read_time=73734623i,io_time=34296428i,weighted_io_time=85406077i 1512012101000000000
> diskio,name=sdh1,host=bfc2ca632aeb writes=4339833i,read_bytes=6017648107008i,write_bytes=707388743680i,read_time=417140405i,io_time=77715394i,reads=30887027i,write_time=251746257i,weighted_io_time=669007259i,iops_in_progress=0i 1512012101000000000
> diskio,name=sdd1,host=bfc2ca632aeb read_bytes=4404390836736i,write_bytes=1412770566144i,read_time=369484011i,write_time=283836129i,weighted_io_time=653476964i,iops_in_progress=0i,reads=12712988i,io_time=54888325i,writes=5514385i 1512012101000000000
> diskio,name=sdf1,host=bfc2ca632aeb io_time=38954613i,weighted_io_time=90827089i,reads=25731040i,writes=458459i,write_bytes=284882509824i,read_time=74919317i,write_time=15892616i,read_bytes=5428289837568i,iops_in_progress=0i 1512012101000000000
> diskio,name=md1,host=bfc2ca632aeb read_bytes=2308956960768i,read_time=0i,write_time=0i,io_time=0i,iops_in_progress=0i,writes=4747366i,write_bytes=706423319040i,weighted_io_time=0i,reads=19891680i 1512012101000000000
> diskio,name=sdb1,host=bfc2ca632aeb write_time=13459i,weighted_io_time=28509i,reads=10324i,writes=4434i,read_bytes=99661824i,write_bytes=4210176i,read_time=15053i,io_time=17138i,iops_in_progress=0i 1512012101000000000
> diskio,host=bfc2ca632aeb,name=sdd read_bytes=4404391135744i,write_bytes=1412770566144i,io_time=60865467i,reads=12713034i,writes=5786742i,weighted_io_time=660354721i,iops_in_progress=0i,read_time=369484110i,write_time=290710836i 1512012101000000000
> diskio,host=bfc2ca632aeb,name=sdg1 reads=23755148i,read_bytes=4936734251520i,write_bytes=87638777856i,read_time=67372945i,io_time=35393508i,writes=149166i,write_time=7281922i,weighted_io_time=74703313i,iops_in_progress=0i 1512012101000000000
> diskio,name=sdj,host=bfc2ca632aeb io_time=78498359i,iops_in_progress=0i,reads=11437097i,read_bytes=4404391135744i,write_bytes=1412770566144i,read_time=489160267i,writes=5756445i,write_time=465862621i,weighted_io_time=955393298i 1512012101000000000
> diskio,name=md2,host=bfc2ca632aeb write_bytes=224192904192i,write_time=0i,weighted_io_time=0i,reads=14987534i,writes=721508i,read_bytes=1935948515328i,read_time=0i,io_time=0i,iops_in_progress=0i 1512012101000000000
> diskio,name=md3,host=bfc2ca632aeb write_time=0i,io_time=0i,weighted_io_time=0i,writes=624841i,read_bytes=2142830045184i,write_bytes=284869499904i,reads=16364558i,read_time=0i,iops_in_progress=0i 1512012101000000000
> diskio,name=loop1,host=bfc2ca632aeb write_time=22585404i,weighted_io_time=145914455i,reads=2793696i,writes=1673938i,write_bytes=485338742784i,read_time=123326746i,read_bytes=128682512384i,io_time=16493577i,iops_in_progress=0i 1512012101000000000
> diskio,name=sdf,host=bfc2ca632aeb reads=25731086i,write_bytes=284882509824i,weighted_io_time=90880904i,iops_in_progress=0i,writes=460545i,read_bytes=5428290136576i,read_time=74925941i,write_time=15938849i,io_time=38971249i 1512012101000000000
> diskio,name=sdc,host=bfc2ca632aeb write_time=5965705i,weighted_io_time=66198761i,reads=24377125i,read_bytes=5081694525952i,write_bytes=108657520640i,iops_in_progress=0i,writes=180372i,read_time=60206328i,io_time=33150768i 1512012101000000000
> diskio,name=sdc1,host=bfc2ca632aeb write_bytes=108657520640i,read_time=60199722i,io_time=33143557i,weighted_io_time=66172356i,reads=24377079i,read_bytes=5081694226944i,write_time=5946284i,iops_in_progress=0i,writes=179116i 1512012101000000000
> diskio,name=sde1,host=bfc2ca632aeb reads=24597463i,read_bytes=5170722430464i,read_time=73734521i,write_time=11593180i,io_time=34289143i,writes=376729i,write_bytes=224203014144i,weighted_io_time=85378773i,iops_in_progress=0i 1512012101000000000
> diskio,name=sdg,host=bfc2ca632aeb reads=23755194i,read_bytes=4936734550528i,write_bytes=87638777856i,io_time=35412519i,writes=152002i,read_time=67379568i,write_time=7317446i,weighted_io_time=74745583i,iops_in_progress=0i 1512012101000000000
> diskio,name=sdh,host=bfc2ca632aeb write_bytes=707388743680i,write_time=258653377i,weighted_io_time=675919357i,iops_in_progress=0i,reads=30887074i,writes=4604273i,read_bytes=6017649454592i,read_time=417140532i,io_time=83438743i 1512012101000000000
> diskio,name=sdi1,host=bfc2ca632aeb iops_in_progress=0i,write_bytes=3918547959808i,io_time=27024713i,weighted_io_time=1682493701i,read_time=74891163i,write_time=1607603174i,reads=7794212i,writes=10838429i,read_bytes=1971918422016i 1512012101000000000
> diskio,name=loop0,host=bfc2ca632aeb reads=142i,iops_in_progress=0i,read_time=2015i,write_time=5282i,io_time=6597i,weighted_io_time=7297i,writes=390i,read_bytes=2838528i,write_bytes=11598016512i 1512012101000000000
> diskio,name=sdb,host=bfc2ca632aeb write_time=13459i,weighted_io_time=28584i,iops_in_progress=0i,reads=10360i,write_bytes=4210176i,read_time=15128i,writes=4434i,read_bytes=100722688i,io_time=17185i 1512012101000000000
> diskio,name=sdj1,host=bfc2ca632aeb read_time=489160141i,write_time=457321395i,weighted_io_time=946843901i,reads=11437051i,writes=5484088i,read_bytes=4404390836736i,write_bytes=1412770566144i,io_time=71328058i,iops_in_progress=0i 1512012101000000000
> diskio,name=md4,host=bfc2ca632aeb write_time=0i,io_time=0i,reads=15070208i,read_bytes=1972453335040i,write_bytes=108650188288i,iops_in_progress=0i,writes=325933i,read_time=0i,weighted_io_time=0i 1512012101000000000
root@unWOLF:/#

Telegraf 1.4.4 with HOST_MOUNT_PREFIX

Docker command docker run --rm --name telegraf144test --privileged -v /mnt/user/appdata/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc -e HOST_MOUNT_PREFIX=/rootfs telegraf:1.4.4 /usr/bin/telegraf --test

Output

2017/11/30 03:22:37 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.disk, Collection 1
> disk,path=/,device=loop1,fstype=btrfs,host=07396336b457 total=8317063168i,free=7523753984i,used=793309184i,used_percent=9.538333038665217,inodes_total=2030533i,inodes_free=2009602i,inodes_used=20931i 1512012157000000000
> disk,path=/etc/resolv.conf,device=loop1,fstype=btrfs,host=07396336b457 total=8317063168i,free=7523753984i,used=793309184i,used_percent=9.538333038665217,inodes_total=2030533i,inodes_free=2009602i,inodes_used=20931i 1512012157000000000
> disk,path=/etc/hostname,device=loop1,fstype=btrfs,host=07396336b457 total=8317063168i,free=7523753984i,used=793309184i,used_percent=9.538333038665217,inodes_total=2030533i,inodes_free=2009602i,inodes_used=20931i 1512012157000000000
> disk,path=/etc/hosts,device=loop1,fstype=btrfs,host=07396336b457 inodes_free=2009602i,inodes_used=20931i,total=8317063168i,free=7523753984i,used=793309184i,used_percent=9.538333038665217,inodes_total=2030533i 1512012157000000000

Telegraf 1.4.3 without HOST_MOUNT_PREFIX

Docker command docker run --rm --name telegraf143testNoPrefix --privileged -v /mnt/user/appdata/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc telegraf:1.4.3 /usr/bin/telegraf --test

Output

2017/11/30 03:23:36 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.diskio, Collection 1
> diskio,name=sdc1,host=4f5d380f717d reads=24377079i,write_time=5946284i,weighted_io_time=66172356i,iops_in_progress=0i,io_time=33143557i,writes=179116i,read_bytes=5081694226944i,write_bytes=108657520640i,read_time=60199722i 1512012217000000000
> diskio,name=sdf,host=4f5d380f717d read_time=74925941i,io_time=38971249i,weighted_io_time=90880904i,iops_in_progress=0i,reads=25731086i,writes=460545i,read_bytes=5428290136576i,write_bytes=284882509824i,write_time=15938849i 1512012217000000000
> diskio,name=sdf1,host=4f5d380f717d read_time=74919317i,io_time=38954613i,weighted_io_time=90827089i,reads=25731040i,writes=458459i,read_bytes=5428289837568i,write_bytes=284882509824i,write_time=15892616i,iops_in_progress=0i 1512012217000000000
> diskio,host=4f5d380f717d,name=md3 read_time=0i,weighted_io_time=0i,io_time=0i,iops_in_progress=0i,reads=16364558i,writes=624841i,read_bytes=2142830045184i,write_bytes=284869499904i,write_time=0i 1512012217000000000
> diskio,name=sdi,host=4f5d380f717d iops_in_progress=0i,read_time=74891166i,write_time=1611060276i,weighted_io_time=1685954254i,write_bytes=3918561398784i,io_time=30090222i,reads=7794255i,writes=12210447i,read_bytes=1971918651392i 1512012217000000000
> diskio,name=loop0,host=4f5d380f717d reads=142i,writes=390i,io_time=6597i,iops_in_progress=0i,read_bytes=2838528i,write_bytes=11598016512i,read_time=2015i,write_time=5282i,weighted_io_time=7297i 1512012217000000000
> diskio,name=sdb,host=4f5d380f717d read_time=15128i,write_time=13459i,io_time=17185i,reads=10360i,writes=4434i,read_bytes=100722688i,write_bytes=4210176i,weighted_io_time=28584i,iops_in_progress=0i 1512012217000000000
> diskio,name=sdd1,host=4f5d380f717d writes=5516813i,read_time=369700668i,write_time=284007491i,reads=12715082i,read_bytes=4404526766592i,write_bytes=1412908879872i,io_time=54903610i,weighted_io_time=653865274i,iops_in_progress=0i 1512012217000000000
> diskio,name=sde,host=4f5d380f717d writes=378468i,write_bytes=224203014144i,weighted_io_time=85406077i,reads=24597510i,read_time=73734623i,write_time=11620349i,io_time=34296428i,iops_in_progress=0i,read_bytes=5170723778048i 1512012217000000000
> diskio,name=sde1,host=4f5d380f717d reads=24597463i,read_bytes=5170722430464i,read_time=73734521i,write_time=11593180i,weighted_io_time=85378773i,iops_in_progress=0i,writes=376729i,write_bytes=224203014144i,io_time=34289143i 1512012217000000000
> diskio,name=sdg,host=4f5d380f717d reads=23755194i,write_bytes=87638777856i,read_time=67379568i,write_time=7317446i,io_time=35412519i,iops_in_progress=0i,writes=152002i,read_bytes=4936734550528i,weighted_io_time=74745583i 1512012217000000000
> diskio,name=sdh1,host=4f5d380f717d reads=30889196i,write_time=252008130i,weighted_io_time=669556083i,read_time=417422641i,io_time=77731269i,iops_in_progress=40i,writes=4342216i,read_bytes=6017787162112i,write_bytes=707526905856i 1512012217000000000
> diskio,name=sdi1,host=4f5d380f717d writes=10838748i,read_bytes=1971918499840i,write_bytes=3918561398784i,io_time=27024878i,iops_in_progress=0i,reads=7794230i,read_time=74891164i,write_time=1607603381i,weighted_io_time=1682493909i 1512012217000000000
> diskio,name=md2,host=4f5d380f717d read_bytes=1935948515328i,write_time=0i,writes=721508i,write_bytes=224192904192i,read_time=0i,io_time=0i,weighted_io_time=0i,iops_in_progress=0i,reads=14987534i 1512012217000000000
> diskio,name=md5,host=4f5d380f717d writes=96628i,read_time=0i,weighted_io_time=0i,reads=14146328i,read_bytes=1848537799168i,write_bytes=87622718464i,write_time=0i,io_time=0i,iops_in_progress=0i 1512012217000000000
> diskio,name=sdb1,host=4f5d380f717d weighted_io_time=28509i,iops_in_progress=0i,read_bytes=99661824i,read_time=15053i,write_bytes=4210176i,write_time=13459i,io_time=17138i,reads=10324i,writes=4434i 1512012217000000000
> diskio,name=sdd,host=4f5d380f717d write_bytes=1412908879872i,reads=12715128i,writes=5789378i,read_bytes=4404527065600i,weighted_io_time=660749607i,iops_in_progress=0i,read_time=369700767i,write_time=290888774i,io_time=60885975i 1512012217000000000
> diskio,name=sdg1,host=4f5d380f717d read_bytes=4936734251520i,read_time=67372945i,write_time=7281922i,reads=23755148i,writes=149166i,write_bytes=87638777856i,io_time=35393508i,weighted_io_time=74703313i,iops_in_progress=0i 1512012217000000000
> diskio,name=sdh,host=4f5d380f717d io_time=83459834i,weighted_io_time=676474785i,read_time=417422768i,writes=4606864i,read_bytes=6017788509696i,write_bytes=707526905856i,write_time=258921854i,iops_in_progress=40i,reads=30889243i 1512012217000000000
> diskio,name=sdj,host=4f5d380f717d iops_in_progress=41i,reads=11439015i,writes=5759028i,read_bytes=4404527016448i,write_bytes=1412908728320i,read_time=489516346i,write_time=466235314i,io_time=78523259i,weighted_io_time=956127785i 1512012217000000000
> diskio,name=md4,host=4f5d380f717d reads=15070208i,write_time=0i,io_time=0i,iops_in_progress=0i,writes=325933i,read_bytes=1972453335040i,write_bytes=108650188288i,read_time=0i,weighted_io_time=0i 1512012217000000000
> diskio,name=loop1,host=4f5d380f717d reads=2794318i,writes=1677375i,read_bytes=128745529344i,io_time=16519485i,write_bytes=485483511808i,read_time=123330184i,write_time=22622908i,weighted_io_time=145955396i,iops_in_progress=0i 1512012217000000000
> diskio,name=sdc,host=4f5d380f717d reads=24377125i,read_bytes=5081694525952i,write_time=5965705i,weighted_io_time=66198761i,iops_in_progress=0i,writes=180372i,write_bytes=108657520640i,read_time=60206328i,io_time=33150768i 1512012217000000000
> diskio,name=sdj1,host=4f5d380f717d writes=5486463i,read_bytes=4404526717440i,write_bytes=1412908728320i,read_time=489516220i,write_time=457686088i,io_time=71346565i,iops_in_progress=41i,reads=11438969i,weighted_io_time=947570388i 1512012217000000000
> diskio,name=md1,host=4f5d380f717d write_bytes=706560951296i,iops_in_progress=0i,weighted_io_time=0i,reads=19891827i,writes=4750034i,read_bytes=2308960131072i,read_time=0i,write_time=0i,io_time=0i 1512012217000000000
* Plugin: inputs.mem, Collection 1
> mem,host=4f5d380f717d available=10936446976i,used=5784469504i,buffered=163840i,active=6673002496i,available_percent=65.40578675266488,total=16720916480i,free=3677229056i,cached=8510648320i,inactive=5734567936i,used_percent=34.59421324733511 1512012217000000000
* Plugin: inputs.processes, Collection 1
> processes,host=4f5d380f717d zombies=0i,stopped=0i,sleeping=446i,unknown=0i,dead=0i,total_threads=1150i,blocked=0i,running=0i,total=446i,paging=0i 1512012217000000000
* Plugin: inputs.swap, Collection 1
> swap,host=4f5d380f717d used_percent=0,total=0i,used=0i,free=0i 1512012217000000000
> swap,host=4f5d380f717d in=0i,out=0i 1512012217000000000
* Plugin: inputs.snmp, Collection 1
* Plugin: inputs.disk, Collection 1
> disk,path=/boot,device=sdb1,fstype=vfat,host=4f5d380f717d used=8816451584i,used_percent=21.75944627824138,inodes_total=0i,inodes_free=0i,inodes_used=0i,total=42949672960i,free=31701360640i 1512012217000000000
> disk,device=/mnt,fstype=none,host=4f5d380f717d,path=/mnt inodes_free=0i,inodes_used=0i,total=42949672960i,free=31701360640i,used=8816451584i,used_percent=21.75944627824138,inodes_total=0i 1512012217000000000
root@unWOLF:/#

Telegraf 1.4.4 without HOST_MOUNT_PREFIX

Docker command docker run --rm --name telegraf144testNoPrefix --privileged -v /mnt/user/appdata/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc telegraf:1.4.4 /usr/bin/telegraf --test

Output

2017/11/30 03:24:05 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.disk, Collection 1
> disk,fstype=btrfs,host=8274693ecb4e,path=/,device=loop1 total=42949672960i,free=31701364736i,used=8816742400i,used_percent=21.760005644899433,inodes_total=0i,inodes_free=0i,inodes_used=0i 1512012245000000000
> disk,device=rootfs,fstype=rootfs,host=8274693ecb4e,path=/rootfs used_percent=9.538579279430573,inodes_total=2030533i,inodes_free=2009600i,inodes_used=20933i,total=8317063168i,free=7523733504i,used=793329664i 1512012245000000000
> disk,path=/rootfs/boot,device=sdb1,fstype=vfat,host=8274693ecb4e free=15149989888i,used=209936384i,used_percent=1.3667798938768239,inodes_total=0i,inodes_free=0i,inodes_used=0i,total=15359926272i 1512012245000000000
> disk,host=8274693ecb4e,path=/rootfs/mnt,device=rootfs,fstype=rootfs inodes_free=2009600i,inodes_used=20933i,total=8317063168i,free=7523733504i,used=793329664i,used_percent=9.538579279430573,inodes_total=2030533i 1512012245000000000
> disk,fstype=xfs,host=8274693ecb4e,path=/rootfs/mnt/disk1,device=md1 total=2999127797760i,free=1123430567936i,used=1875697229824i,used_percent=62.541423917477864,inodes_total=293026624i,inodes_free=292669784i,inodes_used=356840i 1512012245000000000
> disk,fstype=xfs,host=8274693ecb4e,path=/rootfs/mnt/disk2,device=md2 free=1494033113088i,used=1505094684672i,used_percent=50.18441314158506,inodes_total=293026624i,inodes_free=293014737i,inodes_used=11887i,total=2999127797760i 1512012245000000000
> disk,path=/rootfs/mnt/disk3,device=md3,fstype=xfs,host=8274693ecb4e total=2999127797760i,free=1494415253504i,used=1504712544256i,used_percent=50.17167142326664,inodes_total=293026624i,inodes_free=293021669i,inodes_used=4955i 1512012245000000000
> disk,path=/rootfs/mnt/disk4,device=md4,fstype=xfs,host=8274693ecb4e free=1498240638976i,used=1500887158784i,used_percent=50.04412149108779,inodes_total=293026624i,inodes_free=293013129i,inodes_used=13495i,total=2999127797760i 1512012245000000000
> disk,path=/rootfs/mnt/disk5,device=md5,fstype=xfs,host=8274693ecb4e total=2999127797760i,free=1497535639552i,used=1501592158208i,used_percent=50.06762830612003,inodes_total=293026624i,inodes_free=293005790i,inodes_used=20834i 1512012245000000000
> disk,path=/rootfs/mnt/cache,device=sdi1,fstype=btrfs,host=8274693ecb4e inodes_used=0i,total=128035643392i,free=81507282944i,used=45543329792i,used_percent=35.84660381499697,inodes_total=0i,inodes_free=0i 1512012245000000000
> disk,path=/rootfs/mnt/user0,device=shfs,fstype=fuse.shfs,host=8274693ecb4e total=14995638988800i,free=7107655213056i,used=7887983775744i,used_percent=52.601851655907474,inodes_total=1465133120i,inodes_free=1464725109i,inodes_used=408011i 1512012245000000000
> disk,host=8274693ecb4e,path=/rootfs/mnt/user,device=shfs,fstype=fuse.shfs used=7933527105536i,used_percent=52.46108539270817,inodes_total=1465133120i,inodes_free=1464725109i,inodes_used=408011i,total=15123674632192i,free=7189162496000i 1512012245000000000
> disk,path=/rootfs/mnt/disks/2TB,device=sda1,fstype=fuseblk,host=8274693ecb4e free=1727088148480i,used=273273991168i,used_percent=13.661225922626565,inodes_total=1686707824i,inodes_free=1686695079i,inodes_used=12745i,total=2000362139648i 1512012245000000000
> disk,fstype=btrfs,host=8274693ecb4e,path=/rootfs/etc/libvirt,device=loop0 free=948502528i,used=17170432i,used_percent=1.7780794027824909,inodes_total=0i,inodes_free=0i,inodes_used=0i,total=1073741824i 1512012245000000000
> disk,device=loop1,fstype=btrfs,host=8274693ecb4e,path=/rootfs/var/lib/docker used=8816742400i,used_percent=21.760005644899433,inodes_total=0i,inodes_free=0i,inodes_used=0i,total=42949672960i,free=31701364736i 1512012245000000000
> disk,host=8274693ecb4e,path=/rootfs/var/lib/docker/btrfs,device=loop1,fstype=btrfs inodes_total=0i,inodes_free=0i,inodes_used=0i,total=42949672960i,free=31701364736i,used=8816742400i,used_percent=21.760005644899433 1512012245000000000
> disk,path=/rootfs/var/lib/docker/btrfs/subvolumes/f6c9f9a6607e8372cf3f3b832815c6215bf961ddd65c99436e60b7b2723d5094,device=loop1,fstype=btrfs,host=8274693ecb4e used=8816742400i,used_percent=21.760005644899433,inodes_total=0i,inodes_free=0i,inodes_used=0i,total=42949672960i,free=31701364736i 1512012245000000000
> disk,path=/rootfs/etc,device=rootfs,fstype=rootfs,host=8274693ecb4e total=8317063168i,free=7523733504i,used=793329664i,used_percent=9.538579279430573,inodes_total=2030533i,inodes_free=2009600i,inodes_used=20933i 1512012245000000000
> disk,path=/rootfs/etc/libvirt,device=loop0,fstype=btrfs,host=8274693ecb4e inodes_free=0i,inodes_used=0i,total=1073741824i,free=948502528i,used=17170432i,used_percent=1.7780794027824909,inodes_total=0i 1512012245000000000
> disk,fstype=btrfs,host=8274693ecb4e,path=/etc/resolv.conf,device=loop1 inodes_used=0i,total=42949672960i,free=31701364736i,used=8816742400i,used_percent=21.760005644899433,inodes_total=0i,inodes_free=0i 1512012245000000000
> disk,path=/etc/hostname,device=loop1,fstype=btrfs,host=8274693ecb4e total=42949672960i,free=31701364736i,used=8816742400i,used_percent=21.760005644899433,inodes_total=0i,inodes_free=0i,inodes_used=0i 1512012245000000000
> disk,path=/etc/hosts,device=loop1,fstype=btrfs,host=8274693ecb4e total=42949672960i,free=31701364736i,used=8816742400i,used_percent=21.760005644899433,inodes_total=0i,inodes_free=0i,inodes_used=0i 1512012245000000000
> disk,path=/run/docker.sock,device=rootfs,fstype=rootfs,host=8274693ecb4e used_percent=9.538579279430573,inodes_total=2030533i,inodes_free=2009600i,inodes_used=20933i,total=8317063168i,free=7523733504i,used=793329664i 1512012245000000000
> disk,path=/etc/telegraf/telegraf.conf,device=shfs,fstype=fuse.shfs,host=8274693ecb4e inodes_total=293026624i,inodes_free=292669784i,inodes_used=356840i,total=2999127797760i,free=1123430567936i,used=1875697229824i,used_percent=62.541423917477864 1512012245000000000
> disk,path=/run/utmp,device=rootfs,fstype=rootfs,host=8274693ecb4e inodes_free=2009600i,inodes_used=20933i,total=8317063168i,free=7523733504i,used=793329664i,used_percent=9.538579279430573,inodes_total=2030533i 1512012245000000000

I only pasted the [[inputs.disk]] output into the parts above. If you want the full files of the whole run, here you go:

1.4.3 with HOST_MOUNT_PREFIX 1.4.4 with HOST_MOUNT_PREFIX 1.4.3 without HOST_MOUNT_PREFIX 1.4.4 without HOST_MOUNT_PREFIX

If you need any other info, I'm happy to provide it.

So, what does $(pwd)/telegraf.conf do? I figured out that it sets the path of the config file on the host system, but what does your abbreviation of the path do?

This is just command substituion, and is replaced with the output of pwd, which prints my current working directory. So $(pwd)/telegraf.conf turns into /home/dbn/telegraf.conf or such.

And what is --test for? Is it just for verbose output?

This option tells Telegraf to gather once and print the output to the shell instead of sending them to the outputs. Probably easier to use than sending to a file output.

I think I somewhat understand what is happening. In 1.4.4 we updated gopsutil. It had a relevant change where we switched from looking up values in /etc/mtab to /proc/self/mounts.

On Debian Stretch, I notice that they are symlinked:

$ readlink /etc/mtab 
../proc/self/mounts

I would be interested in if this is true on unRAID as well.

At any rate, I added a device /dev/vdb1 on my system to use in some tests, on the host it is mounted like so:

$ cat /proc/self/mounts | grep vdb
/dev/vdb1 /mnt/vdb1 ext4 rw,relatime,data=ordered 0 0

In the container though, the mountpoint has /rootfs prepended:

$ docker run --rm --privileged --name telegraf -v /home/dbn/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock:ro -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc -e HOST_MOUNT_PREFIX=/rootfs telegraf:1.4.3 cat /rootfs/proc/self/mounts | grep vdb
/dev/vdb1 /rootfs/mnt/vdb1 ext4 rw,relatime,data=ordered 0 0

This is exactly the same for /proc/self/mounts, and /rootfs/etc/mtab and /etc/mtab are just symlinks to these. It seems as if Docker is rewriting these files?

Reading through https://github.com/influxdata/telegraf/issues/2811#issuecomment-302364541, @m4ce had a similar problem. He found that if he bind mounts /proc/1/mounts he can point to it using HOST_ETC to get mountpoint info. I tried bind mounting both /proc/self/mounts and /proc/1/mounts into the container, and even though the files are the same on the host, within the container only /proc/self/mounts is modified:

$ docker run --rm --privileged --name telegraf -v /home/dbn/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock:ro -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/foo/etc -e HOST_MOUNT_PREFIX=/rootfs -v /proc/self/mounts:/foo/etc/mtab telegraf:1.4.3 cat /foo/etc/mtab | grep vdb
/dev/vdb1 /rootfs/mnt/vdb1 ext4 rw,relatime,data=ordered 0 0
$ docker run --rm --privileged --name telegraf -v /home/dbn/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock:ro -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/foo/etc -e HOST_MOUNT_PREFIX=/rootfs -v /proc/1/mounts:/foo/etc/mtab telegraf:1.4.3 cat /foo/etc/mtab | grep vdb
/dev/vdb1 /mnt/vdb1 ext4 rw,relatime,data=ordered 0 0

This leads me to a potential workaround for 1.4.4 with HOST_MOUNT_PREFIX. You should be able to add the bind mount -v /proc/1/mounts:/rootfs/proc/self/mounts along with -e HOST_MOUNT_PREFIX=/rootfs and I think the devices will appear again without the /rootfs prefix. Could you give that a try and see if it helps?

That seems to have done the trick, atleast at first glance. Here's the output for 1.4.4 with your additions:

Docker command docker run --rm --name telegraf144test --privileged -v /mnt/user/appdata/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock -v /proc/1/mounts:/rootfs/proc/self/mounts -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc -e HOST_MOUNT_PREFIX=/rootfs telegraf:1.4.4 /usr/bin/telegraf --test

Output

* Plugin: inputs.disk, Collection 1
> disk,path=/,device=rootfs,fstype=rootfs,host=88511f112cce inodes_free=2009587i,inodes_used=20946i,total=8317063168i,free=7517368320i,used=799694848i,used_percent=9.615110909303123,inodes_total=2030533i 1512053498000000000
> disk,fstype=vfat,host=88511f112cce,path=/boot,device=sdb1 total=15359926272i,free=15149989888i,used=209936384i,used_percent=1.3667798938768239,inodes_total=0i,inodes_free=0i,inodes_used=0i 1512053498000000000
> disk,path=/mnt,device=rootfs,fstype=rootfs,host=88511f112cce used=799694848i,used_percent=9.615110909303123,inodes_total=2030533i,inodes_free=2009587i,inodes_used=20946i,total=8317063168i,free=7517368320i 1512053498000000000
> disk,host=88511f112cce,path=/mnt/disk1,device=md1,fstype=xfs inodes_free=292669764i,inodes_used=356860i,total=2999127797760i,free=1118823944192i,used=1880303853568i,used_percent=62.695022698678216,inodes_total=293026624i 1512053498000000000
> disk,path=/mnt/disk2,device=md2,fstype=xfs,host=88511f112cce inodes_free=293014737i,inodes_used=11887i,total=2999127797760i,free=1494033113088i,used=1505094684672i,used_percent=50.18441314158506,inodes_total=293026624i 1512053498000000000
> disk,host=88511f112cce,path=/mnt/disk3,device=md3,fstype=xfs inodes_free=293021669i,inodes_used=4955i,total=2999127797760i,free=1494415253504i,used=1504712544256i,used_percent=50.17167142326664,inodes_total=293026624i 1512053498000000000
> disk,host=88511f112cce,path=/mnt/disk4,device=md4,fstype=xfs inodes_total=293026624i,inodes_free=293013129i,inodes_used=13495i,total=2999127797760i,free=1498240634880i,used=1500887162880i,used_percent=50.04412162766083 1512053498000000000
> disk,path=/mnt/disk5,device=md5,fstype=xfs,host=88511f112cce free=1497535668224i,used=1501592129536i,used_percent=50.067627350108744,inodes_total=293026624i,inodes_free=293005791i,inodes_used=20833i,total=2999127797760i 1512053498000000000
> disk,host=88511f112cce,path=/mnt/cache,device=sdi1,fstype=btrfs used=53070471168i,used_percent=41.76530893541392,inodes_total=0i,inodes_free=0i,inodes_used=0i,total=128035643392i,free=73997836288i 1512053498000000000
> disk,path=/mnt/user0,device=shfs,fstype=fuse.shfs,host=88511f112cce total=14995638988800i,free=7103048613888i,used=7892590374912i,used_percent=52.63257124825989,inodes_total=1465133120i,inodes_free=1464725090i,inodes_used=408030i 1512053498000000000
> disk,path=/mnt/user,device=shfs,fstype=fuse.shfs,host=88511f112cce total=15123674632192i,free=7177046450176i,used=7945660846080i,used_percent=52.541259249573294,inodes_total=1465133120i,inodes_free=1464725090i,inodes_used=408030i 1512053498000000000
> disk,path=/etc/libvirt,device=loop0,fstype=btrfs,host=88511f112cce total=1073741824i,free=948502528i,used=17170432i,used_percent=1.7780794027824909,inodes_total=0i,inodes_free=0i,inodes_used=0i 1512053498000000000
> disk,path=/var/lib/docker,device=loop1,fstype=btrfs,host=88511f112cce total=42949672960i,free=31775211520i,used=8736374784i,used_percent=21.565126377530657,inodes_total=0i,inodes_free=0i,inodes_used=0i 1512053498000000000
> disk,device=loop1,fstype=btrfs,host=88511f112cce,path=/var/lib/docker/btrfs inodes_total=0i,inodes_free=0i,inodes_used=0i,total=42949672960i,free=31775211520i,used=8736374784i,used_percent=21.565126377530657 1512053498000000000
> disk,path=/mnt/disks/2TB,device=sda1,fstype=fuseblk,host=88511f112cce inodes_free=1686695079i,inodes_used=12745i,total=2000362139648i,free=1727088148480i,used=273273991168i,used_percent=13.661225922626565,inodes_total=1686707824i 1512053498000000000
> disk,path=/var/run/docker/netns/44f9348f3849,device=nsfs,fstype=nsfs,host=88511f112cce inodes_free=2009587i,inodes_used=20946i,total=8317063168i,free=7517368320i,used=799694848i,used_percent=9.615110909303123,inodes_total=2030533i 1512053498000000000
* Plugin: inputs.system, Collection 1
> system,host=88511f112cce load15=0.21,n_users=1i,n_cpus=4i,load1=0.98,load5=0.41 1512053498000000000
> system,host=88511f112cce uptime=1080959i,uptime_format="12 days, 12:15" 1512053498000000000
* Plugin: inputs.snmp, Collection 1
* Plugin: inputs.net, Collection 1
> net,interface=eth0,host=88511f112cce bytes_recv=0i,packets_recv=0i,drop_in=0i,bytes_sent=0i,packets_sent=0i,err_in=0i,err_out=0i,drop_out=0i 1512053498000000000
> net,host=88511f112cce,interface=all ip_reasmoks=0i,ip_forwarding=1i,ip_defaultttl=64i,tcp_incsumerrors=0i,udp_outdatagrams=0i,ip_inunknownprotos=0i,icmp_outerrors=0i,icmp_outaddrmaskreps=0i,icmp_indestunreachs=0i,udp_sndbuferrors=0i,icmp_intimestampreps=0i,tcp_inerrs=0i,udplite_sndbuferrors=0i,ip_forwdatagrams=0i,icmp_inredirects=0i,icmp_inechos=0i,tcp_passiveopens=0i,udp_rcvbuferrors=0i,ip_reasmfails=0i,icmp_outaddrmasks=0i,tcp_outsegs=0i,udp_incsumerrors=0i,udp_ignoredmulti=0i,udplite_incsumerrors=0i,ip_inhdrerrors=0i,ip_reasmreqds=0i,icmp_outmsgs=0i,udplite_rcvbuferrors=0i,udplite_inerrors=0i,udplite_indatagrams=0i,ip_reasmtimeout=0i,icmp_outechoreps=0i,tcp_rtoalgorithm=1i,tcp_rtomin=200i,tcp_currestab=0i,icmp_intimeexcds=0i,udp_indatagrams=0i,udp_inerrors=0i,ip_outnoroutes=0i,icmp_outredirects=0i,icmp_inechoreps=0i,icmp_outdestunreachs=0i,tcp_activeopens=0i,udplite_ignoredmulti=0i,ip_inaddrerrors=0i,icmp_outtimestampreps=0i,icmp_outtimeexcds=0i,tcp_maxconn=-1i,udp_noports=0i,ip_fragcreates=0i,ip_outrequests=0i,tcp_rtomax=120000i,icmp_incsumerrors=0i,icmp_inerrors=0i,icmp_outechos=0i,tcp_outrsts=0i,icmp_inaddrmaskreps=0i,tcp_insegs=0i,ip_indelivers=0i,ip_fragoks=0i,ip_inreceives=0i,icmp_insrcquenchs=0i,icmp_outsrcquenchs=0i,icmp_intimestamps=0i,tcp_estabresets=0i,tcp_retranssegs=0i,udplite_noports=0i,icmp_inmsgs=0i,icmp_outparmprobs=0i,icmp_inparmprobs=0i,ip_fragfails=0i,ip_indiscards=0i,ip_outdiscards=0i,icmp_inaddrmasks=0i,icmp_outtimestamps=0i,tcp_attemptfails=0i,udplite_outdatagrams=0i 1512053498000000000
* Plugin: inputs.diskio, Collection 1
> diskio,name=sdd,host=88511f112cce read_bytes=4412453018112i,write_time=292226685i,io_time=61288555i,weighted_io_time=663425300i,iops_in_progress=0i,reads=12745938i,writes=5840267i,write_bytes=1421198749696i,read_time=371038352i 1512053498000000000
> diskio,name=sdd1,host=88511f112cce write_time=285258434i,io_time=55223840i,iops_in_progress=0i,read_bytes=4412452719104i,write_bytes=1421198749696i,read_time=371038253i,reads=12745892i,writes=5564090i,weighted_io_time=656454001i 1512053498000000000
> diskio,name=sde1,host=88511f112cce io_time=34349790i,iops_in_progress=0i,read_time=73802003i,write_time=11593222i,weighted_io_time=85446298i,reads=24599390i,writes=376737i,read_bytes=5170748714496i,write_bytes=224203067392i 1512053498000000000
> diskio,name=sdi1,host=88511f112cce read_time=74912646i,write_time=1618505625i,io_time=27200419i,iops_in_progress=0i,weighted_io_time=1693417553i,reads=7805426i,writes=10977998i,read_bytes=1972219863040i,write_bytes=3949904998400i 1512053498000000000
> diskio,host=88511f112cce,name=sdb1 io_time=17582i,weighted_io_time=29116i,iops_in_progress=0i,reads=10641i,read_bytes=100370432i,read_time=15410i,write_time=13709i,writes=4579i,write_bytes=4284416i 1512053498000000000
> diskio,name=sdb,host=88511f112cce writes=4579i,read_time=15485i,weighted_io_time=29191i,reads=10677i,read_bytes=101431296i,write_bytes=4284416i,write_time=13709i,io_time=17629i,iops_in_progress=0i 1512053498000000000
> diskio,name=sdf,host=88511f112cce weighted_io_time=90931056i,iops_in_progress=0i,reads=25731358i,writes=460608i,write_bytes=284883066880i,io_time=39019014i,read_bytes=5428292635136i,read_time=74975144i,write_time=15939799i 1512053498000000000
> diskio,name=sdf1,host=88511f112cce reads=25731312i,writes=458518i,read_bytes=5428292336128i,read_time=74968520i,io_time=39002331i,weighted_io_time=90877194i,write_bytes=284883066880i,write_time=15893519i,iops_in_progress=0i 1512053498000000000
> diskio,name=sdh,host=88511f112cce writes=4657480i,read_time=418887803i,write_time=260352033i,weighted_io_time=679366404i,reads=30941425i,read_bytes=6026208394752i,write_bytes=715815309312i,io_time=83934084i,iops_in_progress=0i 1512053498000000000
> diskio,name=sdh1,host=88511f112cce reads=30941378i,read_time=418887676i,io_time=78125323i,iops_in_progress=0i,writes=4389286i,read_bytes=6026207047168i,write_bytes=715815309312i,write_time=253353215i,weighted_io_time=672362446i 1512053498000000000
> diskio,name=sdi,host=88511f112cce iops_in_progress=0i,reads=7805451i,writes=12373966i,read_bytes=1972220014592i,write_bytes=3949904998400i,read_time=74912648i,weighted_io_time=1696921369i,write_time=1622005997i,io_time=30308606i 1512053498000000000
> diskio,name=sdj,host=88511f112cce reads=11468355i,writes=5809859i,read_time=491522093i,io_time=79014418i,weighted_io_time=960559933i,read_bytes=4412453018112i,write_bytes=1421198749696i,write_time=468665283i,iops_in_progress=0i 1512053498000000000
> diskio,name=loop1,host=88511f112cce read_bytes=129124986880i,write_bytes=488199045120i,read_time=123486694i,write_time=22972630i,weighted_io_time=146461617i,reads=2803501i,writes=1722733i,io_time=16778938i,iops_in_progress=0i 1512053498000000000
> diskio,name=md4,host=88511f112cce write_bytes=108650566656i,weighted_io_time=0i,iops_in_progress=0i,read_time=0i,write_time=0i,io_time=0i,reads=15070931i,writes=326015i,read_bytes=1972461244416i 1512053498000000000
> diskio,name=sdc,host=88511f112cce writes=180483i,read_bytes=5081702877696i,write_bytes=108658073600i,read_time=60276989i,weighted_io_time=66270916i,iops_in_progress=0i,reads=24377893i,io_time=33222395i,write_time=5967199i 1512053498000000000
> diskio,name=sdc1,host=88511f112cce reads=24377847i,writes=179198i,read_bytes=5081702578688i,io_time=33215184i,weighted_io_time=66244511i,iops_in_progress=0i,write_bytes=108658073600i,read_time=60270383i,write_time=5947778i 1512053498000000000
> diskio,name=sdg,host=88511f112cce reads=23756173i,writes=152106i,weighted_io_time=74817368i,io_time=35483699i,iops_in_progress=0i,read_bytes=4936746621440i,write_bytes=87639232512i,read_time=67450499i,write_time=7318299i 1512053498000000000
> diskio,name=sdj1,host=88511f112cce read_time=491521967i,io_time=71738861i,reads=11468309i,writes=5533682i,read_bytes=4412452719104i,write_bytes=1421198749696i,write_time=460011828i,weighted_io_time=951898116i,iops_in_progress=0i 1512053498000000000
> diskio,name=md3,host=88511f112cce read_time=0i,write_time=0i,io_time=0i,weighted_io_time=0i,writes=624924i,read_bytes=2142832027648i,iops_in_progress=0i,reads=16364788i,write_bytes=284870012928i 1512053498000000000
> diskio,name=loop0,host=88511f112cce weighted_io_time=7297i,iops_in_progress=0i,reads=142i,read_time=2015i,write_time=5282i,io_time=6597i,writes=390i,read_bytes=2838528i,write_bytes=11598016512i 1512053498000000000
> diskio,name=sdg1,host=88511f112cce iops_in_progress=0i,reads=23756127i,read_bytes=4936746322432i,write_bytes=87639232512i,read_time=67443876i,io_time=35464688i,weighted_io_time=74775098i,writes=149240i,write_time=7282775i 1512053498000000000
> diskio,name=md1,host=88511f112cce reads=19914469i,writes=4790311i,write_bytes=714832618496i,io_time=0i,weighted_io_time=0i,iops_in_progress=0i,read_bytes=2309459548160i,read_time=0i,write_time=0i 1512053498000000000
> diskio,name=md2,host=88511f112cce read_bytes=1935974833152i,iops_in_progress=0i,reads=14989551i,write_bytes=224192939008i,read_time=0i,write_time=0i,io_time=0i,weighted_io_time=0i,writes=721518i 1512053498000000000
> diskio,name=md5,host=88511f112cce reads=14147293i,read_bytes=1848549572096i,write_bytes=87622974976i,write_time=0i,writes=96699i,read_time=0i,io_time=0i,weighted_io_time=0i,iops_in_progress=0i 1512053498000000000
> diskio,host=88511f112cce,name=sde write_bytes=224203067392i,read_time=73802105i,write_time=11620391i,weighted_io_time=85473602i,iops_in_progress=0i,reads=24599437i,read_bytes=5170750062080i,writes=378479i,io_time=34357075i 1512053498000000000

So this probably broke for most systems some time ago, but everyone has been using @mc4e's workaround. Now in 1.4.4 the workaround being used also needs modified:

- -v /proc/1/mounts:/foo/etc/mtab -e HOST_ETC=/foo/etc
+ -v /proc/1/mounts:/rootfs/proc/self/mounts -e HOST_ETC=/rootfs/etc

I think then it makes sense to adjust the way HOST_MOUNT_PREFIX works to be more similar to in #2812. Instead of adding the prefix to the paths, it should just remove it after reading the regular /rootfs/proc/self/mounts file. I'll pick up the discussion over on that issue.

Hi,

Reviving this old issue - I didn't find a new one that addresses this. Running on Unraid the above on 1.4.4, I am getting:

docker run --rm --name telegraf144test --privileged -v /mnt/user/appdata/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock -v /proc/1/mounts:/rootfs/proc/self/mounts -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc -e HOST_MOUNT_PREFIX=/rootfs telegraf:1.4.4 /usr/bin/telegraf --test
2024/01/09 05:39:42 I! Using config file: /etc/telegraf/telegraf.conf
* Plugin: inputs.disk, Collection 1
> disk,path=/,device=rootfs,fstype=rootfs,host=1653bee0770e used=503095296i,used_percent=6.072543624808111,inodes_total=2022645i,inodes_free=2005295i,inodes_used=17350i,total=8284753920i,free=7781658624i 1704778783000000000
> disk,path=/boot,device=sda1,fstype=vfat,host=1653bee0770e total=30749130752i,free=29794516992i,used=954613760i,used_percent=3.104522751225771,inodes_total=0i,inodes_free=0i,inodes_used=0i 1704778783000000000
> disk,path=/mnt,device=rootfs,fstype=rootfs,host=1653bee0770e inodes_free=2005295i,inodes_used=17350i,total=8284753920i,free=7781658624i,used=503095296i,used_percent=6.072543624808111,inodes_total=2022645i 1704778783000000000
> disk,path=/mnt/disk1,device=md1p1,fstype=xfs,host=1653bee0770e used_percent=60.2398921119271,inodes_total=390701824i,inodes_free=390380496i,inodes_used=321328i,total=3998833471488i,free=1589940502528i,used=2408892968960i 1704778783000000000
> disk,path=/mnt/disk2,device=md2p1,fstype=xfs,host=1653bee0770e free=1971832373248i,used=2027001098240i,used_percent=50.689810233226225,inodes_total=390701824i,inodes_free=390298258i,inodes_used=403566i,total=3998833471488i 1704778783000000000
> disk,path=/mnt/disk3,device=md3p1,fstype=xfs,host=1653bee0770e inodes_used=5552i,total=3998833471488i,free=2803861454848i,used=1194972016640i,used_percent=29.883015263332304,inodes_total=390701824i,inodes_free=390696272i 1704778783000000000
> disk,path=/mnt/disk4,device=md4p1,fstype=xfs,host=1653bee0770e inodes_free=390701813i,inodes_used=11i,total=3998833471488i,free=3432974557184i,used=565858914304i,used_percent=14.150599627081723,inodes_total=390701824i 1704778783000000000
> disk,path=/mnt/disk5,device=md5p1,fstype=xfs,host=1653bee0770e total=3998833471488i,free=3970919313408i,used=27914158080i,used_percent=0.6980575280023578,inodes_total=390701824i,inodes_free=390701821i,inodes_used=3i 1704778783000000000
> disk,path=/mnt/disk7,device=md7p1,fstype=xfs,host=1653bee0770e used_percent=0.6980575280023578,inodes_total=390701824i,inodes_free=390701821i,inodes_used=3i,total=3998833471488i,free=3970919313408i,used=27914158080i 1704778783000000000
> disk,device=md8p1,fstype=xfs,host=1653bee0770e,path=/mnt/disk8 inodes_total=390701824i,inodes_free=390701821i,inodes_used=3i,total=3998833471488i,free=3970919313408i,used=27914158080i,used_percent=0.6980575280023578 1704778783000000000
> disk,path=/mnt/user0,device=shfs,fstype=fuse.shfs,host=1653bee0770e inodes_total=2734912768i,inodes_free=2734182302i,inodes_used=730466i,total=27991834300416i,free=21711366828032i,used=6280467472384i,used_percent=22.436784259939202 1704778783000000000
> disk,host=1653bee0770e,path=/mnt/user,device=shfs,fstype=fuse.shfs used=6280467472384i,used_percent=22.436784259939202,inodes_total=2734912768i,inodes_free=2734182302i,inodes_used=730466i,total=27991834300416i,free=21711366828032i 1704778783000000000
> disk,path=/var/lib/docker,device=loop2,fstype=btrfs,host=1653bee0770e total=536870912000i,free=525346643968i,used=11078574080i,used_percent=2.0652597430661204,inodes_total=0i,inodes_free=0i,inodes_used=0i 1704778783000000000
> disk,fstype=btrfs,host=1653bee0770e,path=/var/lib/docker/btrfs,device=loop2 free=525346643968i,used=11078574080i,used_percent=2.0652597430661204,inodes_total=0i,inodes_free=0i,inodes_used=0i,total=536870912000i 1704778783000000000
> disk,host=1653bee0770e,path=/etc/libvirt,device=loop3,fstype=btrfs used_percent=0.45017542652724274,inodes_total=0i,inodes_free=0i,inodes_used=0i,total=1073741824i,free=948342784i,used=4288512i 1704778783000000000
> disk,host=1653bee0770e,path=/mnt/remotes/NUC_docker,device=//NUC/docker,fstype=cifs inodes_total=0i,inodes_free=0i,inodes_used=0i,total=984297975808i,free=747471761408i,used=236826214400i,used_percent=24.060418716760218 1704778783000000000
> disk,path=/run/docker/netns/0309be2c25e9,device=nsfs,fstype=nsfs,host=1653bee0770e total=33554432i,free=32067584i,used=1486848i,used_percent=4.43115234375,inodes_total=2026835i,inodes_free=2025870i,inodes_used=965i 1704778783000000000
* Plugin: inputs.hddtemp, Collection 1

Which seems to be working. However, with latest I am getting:

docker run --rm --name telegraf144test --privileged -v /mnt/user/appdata/telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro -v /var/run/utmp:/var/run/utmp:ro -v /:/rootfs:ro -v /sys:/rootfs/sys:ro -v /etc:/rootfs/etc:ro -v /proc:/rootfs/proc:ro -v /var/run/docker.sock:/var/run/docker.sock -v /proc/1/mounts:/rootfs/proc/self/mounts -e HOST_PROC=/rootfs/proc -e HOST_SYS=/rootfs/sys -e HOST_ETC=/rootfs/etc -e HOST_MOUNT_PREFIX=/rootfs telegraf:latest /usr/bin/telegraf --test
2024-01-09T05:40:18Z I! Loading config: /etc/telegraf/telegraf.conf
2024-01-09T05:40:18Z I! Starting Telegraf 1.29.1 brought to you by InfluxData the makers of InfluxDB
2024-01-09T05:40:18Z I! Available plugins: 241 inputs, 9 aggregators, 30 processors, 24 parsers, 60 outputs, 6 secret-stores
2024-01-09T05:40:18Z I! Loaded inputs: disk hddtemp
2024-01-09T05:40:18Z I! Loaded aggregators: 
2024-01-09T05:40:18Z I! Loaded processors: 
2024-01-09T05:40:18Z I! Loaded secretstores: 
2024-01-09T05:40:18Z W! Outputs are not used in testing mode!
2024-01-09T05:40:18Z I! Tags enabled: host=daf61b5a6e34
2024-01-09T05:40:18Z E! [inputs.hddtemp] Error in plugin: dial tcp 127.0.0.1:7634: connect: connection refused
> disk,device=rootfs,fstype=rootfs,host=daf61b5a6e34,mode=rw,path=/ free=7781658624i,inodes_free=2005295i,inodes_total=2022645i,inodes_used=17350i,inodes_used_percent=0.8577876987805572,total=8284753920i,used=503095296i,used_percent=6.072543624808111 1704778819000000000
> disk,device=sda1,fstype=vfat,host=daf61b5a6e34,mode=rw,path=/boot free=29794516992i,inodes_free=0i,inodes_total=0i,inodes_used=0i,inodes_used_percent=0,total=30749130752i,used=954613760i,used_percent=3.104522751225771 1704778819000000000
> disk,device=rootfs,fstype=rootfs,host=daf61b5a6e34,mode=rw,path=/mnt free=7781658624i,inodes_free=2005295i,inodes_total=2022645i,inodes_used=17350i,inodes_used_percent=0.8577876987805572,total=8284753920i,used=503095296i,used_percent=6.072543624808111 1704778819000000000
> disk,device=loop2,fstype=btrfs,host=daf61b5a6e34,mode=rw,path=/var/lib/docker free=525346643968i,inodes_free=0i,inodes_total=0i,inodes_used=0i,inodes_used_percent=0,total=536870912000i,used=11078541312i,used_percent=2.06525376063715 1704778819000000000
> disk,device=loop2,fstype=btrfs,host=daf61b5a6e34,mode=rw,path=/var/lib/docker/btrfs free=525346643968i,inodes_free=0i,inodes_total=0i,inodes_used=0i,inodes_used_percent=0,total=536870912000i,used=11078541312i,used_percent=2.06525376063715 1704778819000000000
> disk,device=loop3,fstype=btrfs,host=daf61b5a6e34,mode=rw,path=/etc/libvirt free=948342784i,inodes_free=0i,inodes_total=0i,inodes_used=0i,inodes_used_percent=0,total=1073741824i,used=4288512i,used_percent=0.45017542652724274 1704778819000000000
2024-01-09T05:40:18Z E! [telegraf] Error running agent: input plugins recorded 1 errors

So basically, only rootfs is being reported, and not a HDDs breakdown per disk. from inside telegraf container - /proc/1/mounts contents are: https://pastebin.com/krPqsvmA /proc/self/mounts contains: https://pastebin.com/73MrG69t

@danielnelson any ideas?

influxdata / telegraf