podman ps --format='{{json .Labels}}' incompatible with docker CLI output (string vs map of strings)

[Romain-Geissler-1A]

Issue Description

The output of podman ps --format='{{json .Labels}}' is incompatible with the Docker CLI output. Docker prints a unique json string, with comma separated key=value, while podman outputs a json map of objects.

Steps to reproduce the issue

Steps to reproduce the issue

1. Start an upstream podman container:

> podman run -t -i --rm --privileged --pull=always -v /shared-volume --name podman-server --hostname=podman-server quay.io/podman/upstream

2. Inside this podman server container, start a podman service socket and a sleeping fedora container:

[root@podman-server /]# podman system service -t 0 unix:///shared-volume/podman.sock &
[1] 13
[root@podman-server /]# podman run -d --rm quay.io/fedora/fedora sleep 9999999
Trying to pull quay.io/fedora/fedora:latest...
Getting image source signatures
Copying blob 718a00fe3212 done   |
Copying config 368a084ba1 done   |
Writing manifest to image destination
75d9bb0e8265d99a56ce0535040601674c69c12166159bb5f5f2055ef25e64c5

3. Try to show the labels of this container using podman, it outputs a json object:

[root@podman-server /]# podman ps --format='{{json .Labels}}'
{"license":"MIT","name":"fedora","vendor":"Fedora Project","version":"39"}

4. From outside of the podman server container, start another container with the docker CLI installed, and mount the same shared volume to run a similar ps command. The output this time is a string:

> podman run -t -i --rm --volumes-from podman-server --pull=always -e DOCKER_HOST=unix:///shared-volume/podman.sock docker.io/docker:cli docker ps --format '{{json .Labels}}'
"version=39,license=MIT,name=fedora,vendor=Fedora Project"

Describe the results you received

podman ps --format='{{json .Labels}}' and docker ps --format='{{json .Labels}}' output differs (different json type).

Describe the results you expected

Well... normally podman aims for docker compatibility...

Here to be honest, I feel the podman output is much more logical, and Docker did a very bad choice to expose a comma separated key-value list... However Docker was there first, and is the de-facto standard so... should podman behave exactly like Docker in this case ? Should podman keep Labels exactly like docker does and expose a new LabelsAsObject field to have both formats available ?

podman info output

Tested using latest `quay.io/podman/upstream` image.

Podman in a container

Yes

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

No response

Additional information

No response

[rhatdan]

Care to open a PR, need to get this out soon, since it would be a breaking change.

[Romain-Geissler-1A]

If it's ok to break podman 5.0 vs podman 4.x output, then ok, I can try to submit this quickly.

[rhatdan]

The more I look at this, the less I like it though. I think our method makes more sense.

Is this causing something to break or just noticing it?

[rhatdan]

@mheon @vrothberg @Luap99 WDYT?

[mheon]

Generally speaking we try to match Docker on these things, but --format is where that begins to stretch. Since it gives an inner view of the data structures, it means that we have to match our data structures exactly to Docker to match output, which is something I'm not keen on.

Honestly, I'd just as soon tag this 6.0 and let us debate it in a year.

[Romain-Geissler-1A]

Is this causing something to break or just noticing it?

No one ever came to me internally to report me this difference. I just noticed today while writing a cleanup script for our Jenkins nodes, currently based on Docker deamon but that I wish to eventually migrate to podman. Given our long standing dual docker/podman usage in Amadeus I always try to write OCI engine agnostic scripts. So no, nothing broken that I know of, for me it affects new code I haven't put in production yet.

[vrothberg]

Let's investigate older versions of Docker. Maybe Docker's output has changed in the past. In such a case, I wouldn't bother too much.

[Romain-Geissler-1A]

I still have the folder I manage in Amadeus with all docker CLI version we have used over these last years, here are the results:

[root@2550c09319c6 /]# for i in /docker-cli/*.*; do echo "$i: $($i/docker ps --format='{{json .Labels}}')"; done
/docker-cli/20.10.12: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/20.10.13: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/20.10.14: "vendor=Fedora Project,version=39,license=MIT,name=fedora"
/docker-cli/20.10.15: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/20.10.16: "version=39,license=MIT,name=fedora,vendor=Fedora Project"
/docker-cli/20.10.17: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/20.10.18: "version=39,license=MIT,name=fedora,vendor=Fedora Project"
/docker-cli/20.10.19: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/20.10.20: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/20.10.21: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/20.10.22: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/20.10.23: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/23.0.0: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/23.0.1: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/23.0.2: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/23.0.3: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/23.0.4: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/23.0.5: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/23.0.6: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/24.0.1: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/24.0.2: "vendor=Fedora Project,version=39,license=MIT,name=fedora"
/docker-cli/24.0.4: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/24.0.5: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/24.0.6: "version=39,license=MIT,name=fedora,vendor=Fedora Project"
/docker-cli/24.0.7: "version=39,license=MIT,name=fedora,vendor=Fedora Project"
/docker-cli/25.0.0: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/25.0.1: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/25.0.2: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/25.0.3: "license=MIT,name=fedora,vendor=Fedora Project,version=39"

And I also found another folder of older docker clients:

[root@2550c09319c6 /]# for i in /docker-cli/TO_BE_DELETED/*/docker; do echo "$i: $($i/docker ps --format='{{json .Labels}}')"; done
/docker-cli/TO_BE_DELETED/17.12.1-ce/docker: "license=MIT,name=fedora,vendor=Fedora Project,version=39"
/docker-cli/TO_BE_DELETED/19.03.2/docker: "version=39,license=MIT,name=fedora,vendor=Fedora Project"
/docker-cli/TO_BE_DELETED/20.10.8/docker: "name=fedora,vendor=Fedora Project,version=39,license=MIT"

So I would say Docker has been rather stable about this since 2017.

[vrothberg]

great work, @Romain-Geissler-1A !

So I would say Docker has been rather stable about this since 2017.

That looks very stable to me as well :)

[Luap99]

Fixing this one case is trivial, in general the way to implement is using a Method called Labels() on the list type the overwrites the behaviour. This stuff doesn't really need to and should influence internal data structures. You can see how docker does it: https://github.com/docker/cli/blob/efb9206433fdb9a24e459d0f7b7cfc4e65b9b14b/cli/command/formatter/container.go#L246

I am fine with changing this one for 5.0 if we want to but in general this is a much bigger problem. Nobody really ever checked most of the --format fields against docker so there are a lot of differences. IMO fixing them one by one is likely not sustainable as they always can mean breaking changes for podman users. So either we give up on docker compat here or commit to actually going through them and test them, of course not possible in time for 5.0. This would be a lot of work and given how little the --format thing is used by people I doubt many care and I don't think it is worth it.

IMO it is important to keep --format types as stable as possible as they are used most often in scripts but I don't think chasing docker compatibility is worth it here.

[Romain-Geissler-1A]

For this specific case, in the end I will just write my code differently in order not to rely on --format (instead I will issue multiple docker/podman ps --filter=label=SomeLabel ) so that it works correctly on all sort of docker and podman client versions.

Now I guess it's a bit too late to align in general the output of --format with docker, maybe for 6.0. Still, in all cases some people will be "broken" (or maybe just a bit bothered, here nothing is broken for me in the end): existing podman users if you decide to eventually change the format, or people like me who try to write container engine agnostic code. I would say, most of the --format found in "ps" or "inspect" are definitely internal details of the OCI engine. Yet I would expect that for the things which are specified by the OCI spec (typically the labels) the different OCI engine looking at docker-cli compatibility should try to find a common format.

I guess there is today no OCI initiative which aims at specifying what a container engine CLI should look like (like in the past the image format/registry protocol has been standardized) ?

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.