oijkn / Docker-Raspberry-PI-Monitoring

A docker-compose stack solution for monitoring host and containers with Prometheus, Grafana, cAdvisor and NodeExporter.
MIT License
363 stars 55 forks source link

Cadvisor fails on Raspberry PiOS Bullseye - cgroup v2 error #10

Closed bdollerup closed 1 year ago

bdollerup commented 2 years ago

Hi there,

I'm trying to get the cadvisor container to work on my RPI 4 with Bullseye Lite. And I'm seeing the below in the container log:

I0102 22:17:36.643729       1 manager.go:1193] Exiting thread watching subcontainers
I0102 22:17:36.643803       1 manager.go:403] Exiting global housekeeping thread
I0102 22:17:36.643870       1 cadvisor.go:247] Exiting given signal: terminated
W0102 22:18:09.391361       1 manager.go:159] Cannot detect current cgroup on cgroup v2
W0102 22:18:11.344404       1 sysinfo.go:203] Nodes topology is not available, providing CPU topology
W0102 22:18:11.345095       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu0/online: open /sys/devices/system/cpu/cpu0/online: no such file or directory
W0102 22:18:11.345338       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu1/online: open /sys/devices/system/cpu/cpu1/online: no such file or directory
W0102 22:18:11.345419       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu2/online: open /sys/devices/system/cpu/cpu2/online: no such file or directory
W0102 22:18:11.345499       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu3/online: open /sys/devices/system/cpu/cpu3/online: no such file or directory
E0102 22:18:11.345768       1 info.go:114] Failed to get system UUID: open /etc/machine-id: no such file or directory
W0102 22:18:11.349130       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu0 online state, skipping
W0102 22:18:11.349208       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu1 online state, skipping
W0102 22:18:11.349359       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu2 online state, skipping
W0102 22:18:11.349502       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu3 online state, skipping
E0102 22:18:11.349528       1 machine.go:72] Cannot read number of physical cores correctly, number of cores set to 0
W0102 22:18:11.350145       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu0 online state, skipping
W0102 22:18:11.350211       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu1 online state, skipping
W0102 22:18:11.350263       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu2 online state, skipping
W0102 22:18:11.350446       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu3 online state, skipping
E0102 22:18:11.350471       1 machine.go:86] Cannot read number of sockets correctly, number of sockets set to 0 

Here's what mount says:

mount | grep cgroup
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime)

Here's my docker-compose file:

version: "3.8"
services:
  cadvisor:
    container_name: cadvisor
    devices:
      - /dev/kmsg
    expose:
      - 8080
    hostname: proxydns-02
    image: zcube/cadvisor:latest
    ipc: shareable
    privileged: true
    restart: unless-stopped
    security_opt:
      - label=disable
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro

I've tried adding the cgroup_enable=memory cgroup_memory=1 option to the /boot/cmdline.txt file, but I'm getting the above error no matter what I do.

Any suggestions are greatly appreciated.

Bjarne

skyajal commented 2 years ago

@bdollerup have you tried the suggestions that were made in the other thread? While cadvisor may complain about warning and error messages like the following:

W0104 21:10:01.505886       1 sysinfo.go:203] Nodes topology is not available, providing CPU topology
W0104 21:10:01.506328       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu0/online: open /sys/devices/system/cpu/cpu0/online: no such file or directory
W0104 21:10:01.506385       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu1/online: open /sys/devices/system/cpu/cpu1/online: no such file or directory
W0104 21:10:01.506432       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu2/online: open /sys/devices/system/cpu/cpu2/online: no such file or directory
W0104 21:10:01.506477       1 sysfs.go:348] unable to read /sys/devices/system/cpu/cpu3/online: open /sys/devices/system/cpu/cpu3/online: no such file or directory
E0104 21:10:01.506572       1 info.go:114] Failed to get system UUID: open /etc/machine-id: no such file or directory
W0104 21:10:01.506827       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu0 online state, skipping
W0104 21:10:01.506862       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu1 online state, skipping
W0104 21:10:01.506897       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu2 online state, skipping
W0104 21:10:01.506932       1 machine.go:253] Cannot determine CPU /sys/bus/cpu/devices/cpu3 online state, skipping
E0104 21:10:01.506942       1 machine.go:72] Cannot read number of physical cores correctly, number of cores set to 0

I have not seen issues with the grafana reporting the data correctly. Give the suggestions in that thread a try to see if that helps.

oijkn commented 1 year ago

This issue has been inactive for some time. Please let me know if it's still relevant and need attention or I will close it.