cadvisor "Cannot detect current cgroup on cgroup v2"

[stefanpoensgen]

I'm currently testing the agent to gather all my metrics and logs. At first it looks to work but the cadvisor part is reporting "Cannot detect current cgroup on cgroup v2" Any tipps how to solve this?

docker-compose.yml looks like

    agent:
        image: grafana/agent:latest
        restart: always
        volumes:
            - ./agent/config:/etc/agent-config
            - /var/run/docker.sock:/var/run/docker.sock
            - /var/lib/docker/:/var/lib/docker:ro
            - /proc:/host/proc:ro
            - /sys:/host/sys:ro
            - /:/host/root:ro
            - /:/rootfs:ro
            - /var/run:/var/run:ro
            - /sys:/sys:ro
        entrypoint:
            - /bin/agent
            - -server.http.address=0.0.0.0:12345
            - -config.file=/etc/agent-config/agent.yml
            - -metrics.wal-directory=/tmp/agent/wal
            - -config.expand-env
            - -config.enable-read-api
        devices:
            - /dev/kmsg:/dev/kmsg
        privileged: true

agent.yml looks like

integrations:
  cadvisor:
    docker_only: true
    enabled: true
    instance: localhost:9090
    relabel_configs:
      - action: replace
        replacement: integrations/docker
        target_label: job

logs:
  configs:
    - clients:
        - url: http://10.0.0.2:3100/loki/api/v1/push
      name: integrations
      positions:
        filename: /tmp/positions.yaml
      scrape_configs:
        - docker_sd_configs:
            - host: unix:///var/run/docker.sock
              refresh_interval: 5s
          job_name: integrations/docker
          relabel_configs:
            - action: replace
              replacement: integrations/docker
              source_labels:
                - __meta_docker_container_id
              target_label: job
            - action: replace
              replacement: localhost:9090
              source_labels:
                - __meta_docker_container_id
              target_label: instance
            - regex: /(.*)
              source_labels:
                - __meta_docker_container_name
              target_label: container
            - source_labels:
                - __meta_docker_container_log_stream
              target_label: stream

[ShubhamRasal]

I had the same problem, but it just a warning.

caller=klog.go:108 level=warn integration=cadvisor func=Warningf msg="Cannot detect current cgroup on cgroup v2"

I was still able to get metrics

[h4ckninja]

Adding that I'm getting this too. Setting up the hosted grafana agent on a server and getting this message.

[makwanji]

I am getting this message too. plz share if any solution for this. Thx

[theendie]

Any updates here? I am facing same error. Thanks

[ilsaul]

I find this article but I don't understand complitly: https://serverfault.com/questions/1053187/systemd-fails-to-run-in-a-docker-container-when-using-cgroupv2-cgroupns-priva

[mattdurham]

This may be related to https://github.com/google/cadvisor/issues/3232, we also have updated the dependencies so if you are affected by this give it another try and let us know.

[discapes]

I also see all memory related stats as 0, along with the error message of the title. (even though I have cgroup_enable=memory, on Deb 12). Would be cool to get this working so I can see mem stats for my containers, anyone feel free to ping me if there's a fix.

[jjgmckenzie]

@discapes I had this exact issue and I discoved that I was using gcr.io/cadvisor/cadvisor:latest - which is not the latest version, it's from 2020. What worked for me was using gcr.io/cadvisor/cadvisor:v0.47.2 instead; now memory stats work fine.

[bennlol]

Thank You!!!

@discapes I had this exact issue and I discoved that I was using gcr.io/cadvisor/cadvisor:latest - which is not the latest version, it's from 2020. What worked for me was using gcr.io/cadvisor/cadvisor:v0.47.2 instead; now memory stats work fine.

Thank You!!! This fixed the problem causing my logs to be empty with the error in the title, as well my memory metrics which I had spent a few hours debugging at this point. The version as of writing this post is v0.49.1

[rfratto]

Hi there :wave:

On April 9, 2024, Grafana Labs announced Grafana Alloy, the spirital successor to Grafana Agent and the final form of Grafana Agent flow mode. As a result, Grafana Agent has been deprecated and will only be receiving bug and security fixes until its end-of-life around November 1, 2025.

To make things easier for maintainers, we're in the process of migrating all issues tagged variant/flow to the Grafana Alloy repository to have a single home for tracking issues. This issue is likely something we'll want to address in both Grafana Alloy and Grafana Agent, so just because it's being moved doesn't mean we won't address the issue in Grafana Agent :)

[mattdurham]

Let's add a disclaimer to cadvisor component to reference not using latest.

[IMpcuong]

gcr.io/cadvisor/cadvisor:v0.47.2

@jjgmckenzie you're saving my life, tysm!