Issue Docker Desktop, Cadvisor and WSL

[AlbertoSoutullo]

It looks like if you are using Windows with WSL, using cadvisor can get messy.

When you run the standard recommended cadvisor command, which is like:

sudo docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  --publish=8080:8080 \
  --detach=true \
  --name=cadvisor \
  --privileged \
  --device=/dev/kmsg \
  gcr.io/cadvisor/cadvisor:$VERSION

This might work under a linux distritubion, but this does not work under WSL.

You migth get errors like:

cadvisor | E0823 20:34:34.010493 1 manager.go:1084] Failed to create existing container: /docker/f8b0931d2a1809803906b9e25bb8285438ccabfbfc96e8a7b82cdd29d38a15d9: failed to identify the read-write layer ID for container "f8b0931d2a1809803906b9e25bb8285438ccabfbfc96e8a7b82cdd29d38a15d9". - open /var/lib/docker/image/overlay2/layerdb/mounts/f8b0931d2a1809803906b9e25bb8285438ccabfbfc96e8a7b82cdd29d38a15d9/mount-id: no such file or directory

This error is logged in the cadvisor container. You can still go to localhost:8080 and check that cadvisor looks like it is working, but it cannot show data from the docker containers, since it cannot access to it.

This issue is well explained here, but under my personal case it was a bit different.

As explained in the posted link, cadvisor is expecting to read information under /var/lib/docker. In WSL, this folder is empty. The information that cadvisor is looking for is under \\wsl$\docker-desktop-data\version-pack-data\community\docker which you can access pasting this path in File Explorer. In my personal case, don't know exactly why, it was located in \\wsl$\docker-desktop-data\data\docker. So the only steps you need to do inside WSL are:

• sudo mkdir /mnt/windows_docker (or whatever folder/path you want)
• sudo mount -t drvfs '\\wsl$\docker-desktop-data\version-pack-data\community\docker' /mnt/windows_docker or in my case
• sudo mount -t drvfs '\\wsl$\docker-desktop-data\data\docker' /mnt/windows_docker

And then modify the cadvisor docker run command accordingly with this new information:

docker run \
--volume=/:/rootfs:ro \
--volume=/var/run:/var/run:rw \
--volume=/var/lib/docker/:/var/lib/docker:ro \
--volume=/dev/disk/:/dev/disk:ro \
--volume=/sys:/sys:ro \
--volume=/etc/machine-id:/etc/machine-id:ro \
--volume=/mnt/windows_docker/:/rootfs/var/lib/docker:ro \ # <--- new volume
--publish=8080:8080 \
--detach=true \
--name=cadvisor \
--privileged \
--device=/dev/kmsg \
gcr.io/cadvisor/cadvisor

The error should not appear now, and you will be able to access to the containers information.

Extra information:

Docker Desktop can get crazy sometimes. Engine might not start, or commands that should work suddently they are not working anymore. After extensive usage with lots of containers, docker desktop and kurtosis, I find that sometimes when something is weirdly behaving, it is good to restart the computer (sigh).

It is recommented to delete anything docker related in windows and WSL before trying this steps.

Edit: Well, it looks like the behaviour can still be erratic as hell, one day working fine and the next one it is not. Another solution I found is to mount \\wsl$\docker-desktop-data\data\docker in the home directory in WSL, something like: sudo mount -t drvfs '\\wsl$\docker-desktop-data\data\docker' /home/your_user/windows_docker God knows why, but this works much more consistently.

[ofsaleem]

thanks for this guide, one thing i'm not understanding is why we bind mount the new volume path to /rootfs/var/lib/docker instead of /var/lib/docker

[AlbertoSoutullo]

thanks for this guide, one thing i'm not understanding is why we bind mount the new volume path to /rootfs/var/lib/docker instead of /var/lib/docker

It's been half a year from this so I don't remember exactly. I don't know if it works doing it like that, if it does maybe I just didn't realize about that when I was working with this. Still, glad it was helpful for you!