search

prometheus / node_exporter

Exporter for machine metrics
https://prometheus.io/
Apache License 2.0
10.97k stars 2.33k forks source link

zfs collector in 1.1.0 prints constant warnings on older ZFS versions #1960

Closed siebenmann closed 3 years ago

siebenmann commented 3 years ago

In node exporter version 1.1.0, the ZFS collector has been changed so that it prints a warning to the logs on every scrape if you are using a version of ZFS on Linux without /proc/spl/kstat/zfs/<pool>/state file(s). As far as I can tell, this pool state file feature landed in ZFS on Linux in 0.8.0. The Ubuntu-supplied version of ZFS on Linux in Ubuntu 18.04 is v0.7.5-1ubuntu16.10, which lacks these state files, and so running 1.1.0 on Ubuntu 18.04 with their stock (and supported) ZoL produces a constant stream of warnings (one per scrape, which in practice means one every fifteen or ten seconds or so for many people).

Since these are warnings, you cannot suppress them in the log without losing significant amounts of log messages. Since this is a normal configuration, I believe that it should be a debug message. As it makes using 1.1.0 on an Ubuntu 18.04 ZoL server a problem, I also believe that a fix for this should be rolled out as part of a 1.1.1 bugfix release.

Host operating system: output of uname -a

Linux sanandreas 4.15.0-118-generic #119-Ubuntu SMP Tue Sep 8 12:30:01 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

node_exporter, version 1.1.0 (branch: HEAD, revision: 0e74fbcd5fe3b98246292829a8e81e3133e17033)
  build user:       root@c81c7415c0ee
  build date:       20210205-22:54:09
  go version:       go1.15.8
  platform:         linux/amd64

node_exporter command line flags

node_exporter --no-collector.hwmon --no-collector.cpufreq --no-collector.wifi --collector.netdev.device-exclude='^lo$' --collector.systemd --collector.systemd.unit-include='^.+\.service$' --collector.systemd.unit-exclude='^(user|ifup)@.*$' --collector.ntp --no-collector.pressure --collector.processes --collector.textfile.directory=/var/local/prometheus/node-exporter --collector.filesystem.ignored-mount-points='^/(sys|proc|dev)($|/)' --collector.filesystem.ignored-fs-types='^(autofs|binfmt_misc|cgroup|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|mqueue|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|sysfs|tracefs|rootfs|selinuxfs|nfs)$'

Are you running node_exporter in Docker?

No.

What did you do that produced an error?

Run node exporter version 1.1.0 on an Ubuntu 18.04 machine with the Ubuntu stock ZFS version.

What did you expect to see?

Quiet logs.

What did you see instead?

Constant log messages of: level=warn ts=2021-02-09T01:14:09.074Z caller=zfs_linux.go:125 collector=zfs msg="Not found pool state files".

SuperQ commented 3 years ago

Thanks, you're right, this should be a Debug level log.