1 machine_libipmctl.go:64] There are no NVM devices!

[JPFreeski]

I've tried to solve this by setting:

priveleged: true & ` expose:

• 8080`

But the issue persists. What's causing this. Is it my configuration?

` cadvisor: container_name: mon_cadvisor image: gcr.io/cadvisor/cadvisor:v0.46.0 restart: unless-stopped ports:

• "8080:8080" expose:
• 8080 volumes:
• /:/rootfs:ro
• /var/run:/var/run:ro
• /sys:/sys:ro
• /var/lib/docker/:/var/lib/docker:ro
• /dev/disk/:/dev/disk:ro devices:
• /dev/kmsg:/dev/kmsg priveleged: true`

I wonder if this is causing the issue for me that I can't view any of the containers in grafana. I can see them in the :8080/docker/ web gui without any issue but when I try to add some of the dashboards into grafana the only container I can see is "cadvisor" They appear but I'm unable to select from the grafana drop down menu.

Does anyone have any thoughts on how to solve this?

[frapsky]

+1

[C8opmBM]

+1

[iwankgb]

It seems to me that your problem is unrelated to the error message and that level for this message should be decreased.

Regarding the problem - can you answer following questions:

• what distribution are you running?
• what kernel version are you running?
• what CPU architecture are you using?

[JPFreeski]

Hi iwankgb,

a) So decrease severity for that error message?

b) The VM is running on the following:

Docker Version 20.10.22 Docker API Version 1.41 Kernel Version 5.15.0-57-generic OS Version Ubuntu 22.04.1 LTS

The host is virtualized on xencenter running: Vendor: GenuineIntel Model: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz Speed: 2099 MHz

Let me know if there is anything else that could help.

I'm going to spin up another copy of cadvisor on a different server hosting various containers and see if I come to the same reuslt.

Thanks!

Some more screenshots:

[rdeselle07]

+1

[Rhevin]

+1

monitoring-cadvisor-1       | W0305 06:39:13.258706       1 machine_libipmctl.go:64] There are no NVM devices!
Docker version 20.10.23-ce, build 6051f1429
opensuse micro os
6.2.1-1-default

[guidonguido]

+1

[niinod]

I've got the same problem. The strange thing is, that I had it working earlier today, and I tried adding another cadvisor from another machine to prometheus. I had to change the port because 8080 was already in use on that host, and I got this error. Later I got it on the host that worked earlier.

[manut999]

adding privileged: true and reboot helped for me

[pthoelken]

The problem still exists at my system with Debian 11.7 and Docker version 20.10.5+dfsg1, build 55c4c88

[C8opmBM]

can confirm, still have the issue even with privileged: true setting added.

[jawys]

the problem still exists in my case. dont know how to fix it :/

root@NUC10 ~# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
root@NUC10 ~# docker service logs swarmprom_cadvisor -ft
2023-06-22T14:17:39.082806406Z swarmprom_cadvisor.0.921bfw1vh23q@NUC10    | W0622 14:17:39.078111       1 machine_libipmctl.go:64] There are no NVM devices!
2023-06-22T14:17:55.877375052Z swarmprom_cadvisor.0.921bfw1vh23q@NUC10    | W0622 14:17:55.877317       1 container.go:485] Failed to get RecentStats("/system.slice/docker-de27c3849b33162e5a962466657c718384d702b32e18685fda21f38fb437a61b.scope") while determining the next housekeeping: unable to find data in memory cache
2023-06-22T14:15:53.101060206Z swarmprom_cadvisor.0.iz88imepnb4q@NUC10    | W0622 14:15:53.100910       1 machine_libipmctl.go:64] There are no NVM devices!
2023-06-22T14:16:50.130412640Z swarmprom_cadvisor.0.iz88imepnb4q@NUC10    | W0622 14:16:50.130343       1 container.go:485] Failed to get RecentStats("/system.slice/docker-efbc7c08f2681d582710b51162493748bb7763ca45d2d96973ed7fe4a3007926.scope") while determining the next housekeeping: unable to find data in memory cache
^C⏎
root@NUC10 ~# docker version
Client: Docker Engine - Community
 Version:           23.0.4
 API version:       1.42
 Go version:        go1.19.8
 Git commit:        f480fb1
 Built:             Fri Apr 14 10:32:03 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.4
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.8
  Git commit:       cbce331
  Built:            Fri Apr 14 10:32:03 2023
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[cyrsis]

same in here

[victor-nunesm]

The following works for me:

services:
  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk:/dev/disk:ro
    devices:
      - /dev/kmsg

[Graxo]

The following works for me:

services:
  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk:/dev/disk:ro
    devices:
      - /dev/kmsg

Also works for me! Thanks <3

[kevin-bockman]

Note to others - it's just a warning so it's fine.

Also, you're not supposed to use the latest tag. It "works" on latest because that message isn't there and it's an old version. You're supposed to use fixed version tags.

It'll really be "fixed" once there's a new release.