Podman support

[joachimBurket]

Description Also supporting Podman

Alternatives considered Can only have Podman on the concerned host.

Additional context Tried to use it with podman-docker (which creates a docker script that runs podman, and also links /var/run/docker.sock to /run/podman/podman.sock.

The problem is that the output of the commands (e.g. docker system ps --format "{{json .}}" or docker events --format "{{json .}}") are not exactly the same. Maybe by having structs to also parse Podman outputs, Docuum could also work for Podman.

[stepchowfun]

Thanks @joachimBurket for submitting this feature request! I'd support this if it doesn't substantially increase the complexity of the code.

[joachimBurket]

Seems legit. I've tested the docker commands launched by the code (hope I haven't missed some) to check the output differences with podman. The version of docker used was the 20.10.16, and of podman was the 4.1.1.

• docker image inspect --format {{.ID}} $IMAGE -> same
• docker image inspect --format {{.Parent}} $IMAGE_ID -> same
• docker image ls --all --no-trunc --format "{{.ID}}\\t{{.Repository}}\\t{{.Tag}}\\t{{.CreatedAt}} -> same
• docker container ls --all --no-trunc --format "{{.ID}}" -> same
• docker container inspect --format "{{.Image}}" $CONTAINER_ID -> same
• docker info --format "{{.DockerRootDir}}" -> in Podman, the DockerRootDir is in store.graphRoot
• docker system df --format "{{json .}}" -> The code uses thetypeandsizefields. Podman didn't returned thesizewhen using--format`. I opened an issue in podman repo and a pull request has been merged for that.
• docker image rm --force --no-prune $IMAGE -> podman has no --no-prune argument

• docker events --format "{{json .}}" -> Podman events is the most different command.

  # output of an rmi
  {"ID":"d36e2662aef639867df298eb6ec37544a91dc26057a2a6be057bd9001a9841dc","Name":"docker.io/library/rust:latest","Status":"remove","Time":"2022-07-11T22:39:44.785453695+02:00","Type":"image","Attributes":null}
  
  # output of a pull
  {"Name":"rust:latest","Status":"pull","Time":"2022-07-11T22:37:33.436419221+02:00","Type":"image","Attributes":null}

Do you think it could be manageable? I'm not an experienced rust programmer but I'll try to have a look and test out some things :)

[stepchowfun]

Thanks for the investigation!

I would be willing to review a pull request for this, but I have to put some disclaimers on it:

• If the code starts to get too ugly from dealing with subtle differences between Docker and Podman, I might reject the pull request. So unfortunately you'd be rolling the dice by taking this on.
• There would need to be an integration test validating that this actually works with Podman (we already have one for Docker).
• If Podman ever makes a breaking change that causes Docuum to no longer work with it, I may end up reverting the change / removing Podman support (especially if it's not trivial to fix). I am only officially committing to maintaining Docker support.

Another avenue that might be worth pursuing: if Podman is aiming for compatibility with Docker, could they just close the gap and match what Docker does (e.g., by changing what podman events --format "{{json .}}" outputs)? Then we wouldn't need to make any changes here, and Podman would be able to claim even more compatibility with Docker. That seems like a bigger win for everyone.

[joachimBurket]

Thanks for the details about that. I like the other avenue and will ask on Podman repo if they are willing to match Docker's API, which would indeed be a safer bet. I'll keep you updated

[kjoyce77]

The api compatibility is at the socket level. So you can run docker in a container and share the socket with the container and everything seems to work correctly. ie

docker run \
  --init \
  --rm \
  --tty \
  --name docuum \
  --volume /var/run/podman/podman.sock:/var/run/docker.sock \
  --volume docuum:/root \
  stephanmisc/docuum --threshold '10 GB'

Theoretically you could also install the docker cli on a podman host instead of the podman-docker method. Either way this should be a workaround for anyone that wants podman support. I confirmed the outputs are different.

# podman system df --format "{{json .}}"
{"Type":"Images","Total":2,"Active":2,"RawSize":71779946,"RawReclaimable":0,"TotalCount":2,"Size":"71.78MB","Reclaimable":"0B (0%)"}
{"Type":"Containers","Total":2,"Active":2,"RawSize":516,"RawReclaimable":0,"TotalCount":2,"Size":"516B","Reclaimable":"0B (0%)"}
{"Type":"Local Volumes","Total":0,"Active":0,"RawSize":0,"RawReclaimable":0,"TotalCount":0,"Size":"0B","Reclaimable":"0B (0%)"}

# podman exec docuum docker system df --format "{{json .}}"
{"Active":"2","Reclaimable":"0B","Size":"0B","TotalCount":"2","Type":"Images"}
{"Active":"2","Reclaimable":"0B (0%)","Size":"516B","TotalCount":"2","Type":"Containers"}
{"Active":"0","Reclaimable":"0B","Size":"0B","TotalCount":"0","Type":"Local Volumes"}
{"Active":"0","Reclaimable":"0B","Size":"0B","TotalCount":"0","Type":"Build Cache"}

[stepchowfun]

Hi @kjoyce77, are you saying Docuum now works with Podman (with the Unix domain socket mapping)? This is kind of surprising to me given the differences that @joachimBurket pointed out. Maybe those incompatibilities have been addressed upstream in Podman, and that's why it works now?

[kjoyce77]

Hi @stepchowfun I don't think anything has changed. Look at the output. If you run a native docker executable against the podman socket it works. If you run a docker wrapper around the podman executable the output is different.

[stepchowfun]

Ah I think I understand you. The Podman CLI returns different output than the Docker CLI, but not because the daemon is returning different results—only because the CLIs are different. The Docker socket and the Podman socket are interchangeable. Thanks for pointing this out!

[joachimBurket]

Thanks for the tip @kjoyce77. Regarding the Issue to align podman event cli with docker cli, it is still open, a PR is started, but I'm not really sure it will make docuum work with Podman. For now, I have made a quick and dirty fork of docuum with podman support, which is kind of working on our CI runner (have an issue with the images size, but otherwise it is working). I will definitely try to use docuum like you suggested our CI runner, it seems a more stable approach 😅

[joachimBurket]

Hi @kjoyce77, I tried to run docuum inside a container with the podman socket mounted to /var/run/docker.sock inside the container. Docuum is able to listen to the podman events from inside the container, but the size is always 0B.

podman run \
  --init \
  -it \
  --rm \
  --name docuum \
  --volume /var/run/podman/podman.sock:/var/run/docker.sock --volume /root/docuum:/root \
  --privileged \
  stephanmisc/docuum:0.22.4 --threshold "20 GB"
[2023-08-03 08:43:57 +00:00 INFO] Performing an initial vacuum on startup…
[2023-08-03 08:43:57 +00:00 DEBUG] Docker images are using 0 B, which is within the limit of 20.00 GB.
[2023-08-03 08:43:57 +00:00 INFO] Listening for Docker events…

[2023-08-03 08:46:36 +00:00 DEBUG] Waking up…
[2023-08-03 08:46:36 +00:00 DEBUG] Updating last-used timestamp for image sha256:18c252f1033fc34a27ea8825c2a9946a899eed31589e1f6a0be8d72453ecdb83…
[2023-08-03 08:46:45 +00:00 DEBUG] Docker images are using 0 B, which is within the limit of 20.00 GB.
[2023-08-03 08:46:45 +00:00 DEBUG] Going back to sleep…

podman system df
TYPE           TOTAL       ACTIVE      SIZE        RECLAIMABLE
Images         5           1           850.8MB     784.4MB (92%)
Containers     1           1           0B          0B (0%)
Local Volumes  0           0           0B          0B (0%)

podman exec docuum docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          5         2         0B        0B
Containers      1         1         0B        0B
Local Volumes   0         0         0B        0B
Build Cache     0         0         0B        0B

Do you have the same behavior?

Podman version used: 4.4.1

[kjoyce77]

@joachimBurket I didn't get as far as you yet but looking at my post above I ran the same commands with json and indeed got the same result. This is a work thing that got sidelined temporarily.

It might be worth seeing what podman-remote reports. I don't see how podman-remote wouldn't also report 0b so there is either some podman configuration needed or a bug that needs fixing on the podman side.

[kjoyce77]

@joachimBurket This is back from the sideline from me. I think I am just going to read the state.yml file (.local/share/docuum/state.yml) and then do the deletions through a different script. Although I will see if I can find/open an issue with podman.