Podman fails to recognize HEALTHCHECK in certain BuildKit images

[0xF4CED]

Issue Description

When pulling (some) images that were built with BuiltKit, podman fails to parse/store the declared healtcheck configuration of the dockerimage correctly. Tested with docker.io/adguard/adguardhome@sha256:fc01fa555e6b324378176dff676c7791b2bb6ebfcf63bd2599db568287de7230 This appears to be (almost) identical to the previously resolved issue #12226, possibly indicating a regression or incomplete fix.

Steps to reproduce the issue

podman pull docker.io/adguard/adguardhome@sha256:fc01fa555e6b324378176dff676c7791b2bb6ebfcf63bd2599db568287de7230
podman run --privileged --rm --name dinp --replace -d docker.io/docker
podman exec dinp docker pull docker.io/adguard/adguardhome@sha256:fc01fa555e6b324378176dff676c7791b2bb6ebfcf63bd2599db568287de7230
diff --color <(podman image inspect adguard/adguardhome | grep -i -A3 healthcheck) <(podman exec dinp docker inspect adguard/adguardhome | grep -A8 Healthcheck)

Describe the results you received

<                     "created_by": "HEALTHCHECK &{[\"CMD\" \"/opt/adguardhome/scripts/healthcheck.sh\"] \"30s\" \"10s\" \"0s\" '\\x03'}",
<                     "comment": "buildkit.dockerfile.v0",
<                     "empty_layer": true
<                },
---
>             "Healthcheck": {
>                 "Test": [
>                     "CMD",
>                     "/opt/adguardhome/scripts/healthcheck.sh"
>                 ],
>                 "Interval": 30000000000,
>                 "Timeout": 10000000000,
>                 "Retries": 3
>             },

Describe the results you expected

Podman should properly parse and store the healthcheck that is specified in the dockerimage.

podman info output

Details

```yaml host: arch: amd64 buildahVersion: 1.30.0 cgroupControllers: - cpu - io - memory - pids cgroupManager: systemd cgroupVersion: v2 conmon: package: conmon-2.1.7-2.fc38.x86_64 path: /usr/bin/conmon version: 'conmon version 2.1.7, commit: ' cpuUtilization: idlePercent: 81.04 systemPercent: 6.34 userPercent: 12.63 cpus: 4 databaseBackend: boltdb distribution: distribution: fedora variant: silverblue version: "38" eventLogger: journald hostname: fedora idMappings: gidmap: - container_id: 0 host_id: 1000 size: 1 - container_id: 1 host_id: 100000 size: 65536 uidmap: - container_id: 0 host_id: 1000 size: 1 - container_id: 1 host_id: 100000 size: 65536 kernel: 6.3.7-200.fc38.x86_64 linkmode: dynamic logDriver: journald memFree: 1171042304 memTotal: 8250187776 networkBackend: netavark ociRuntime: name: crun package: crun-1.8.5-1.fc38.x86_64 path: /usr/bin/crun version: |- crun version 1.8.5 commit: b6f80f766c9a89eb7b1440c0a70ab287434b17ed rundir: /run/user/1000/crun spec: 1.0.0 +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL os: linux remoteSocket: exists: true path: /run/user/1000/podman/podman.sock security: apparmorEnabled: false capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT rootless: true seccompEnabled: true seccompProfilePath: /usr/share/containers/seccomp.json selinuxEnabled: true serviceIsRemote: false slirp4netns: executable: /usr/bin/slirp4netns package: slirp4netns-1.2.0-12.fc38.x86_64 version: |- slirp4netns version 1.2.0 commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383 libslirp: 4.7.0 SLIRP_CONFIG_VERSION_MAX: 4 libseccomp: 2.5.3 swapFree: 8249143296 swapTotal: 8249143296 uptime: 0h 8m 56.00s plugins: authorization: null log: - k8s-file - none - passthrough - journald network: - bridge - macvlan - ipvlan volume: - local registries: search: - registry.fedoraproject.org - registry.access.redhat.com - docker.io - quay.io store: configFile: /var/home/podmanuser/.config/containers/storage.conf containerStore: number: 1 paused: 0 running: 0 stopped: 1 graphDriverName: overlay graphOptions: {} graphRoot: /var/home/podmanuser/.local/share/containers/storage graphRootAllocated: 498403901440 graphRootUsed: 44006096896 graphStatus: Backing Filesystem: btrfs Native Overlay Diff: "true" Supports d_type: "true" Using metacopy: "false" imageCopyTmpDir: /var/tmp imageStore: number: 60 runRoot: /run/user/1000/containers transientStore: false volumePath: /var/home/podmanuser/.local/share/containers/storage/volumes version: APIVersion: 4.5.1 Built: 1685123928 BuiltTime: Fri May 26 19:58:48 2023 GitCommit: "" GoVersion: go1.20.4 Os: linux OsArch: linux/amd64 Version: 4.5.1 ```

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

No response

Additional information

When pulling the image with docker, saving the image and loading it into podman, everything works as expected.

podman exec dinp docker save adguard/adguardhome | podman load
podman inspect adguardhome:latest | grep -A8 Healthcheck

"Healthcheck": {
               "Test": [
                    "CMD",
                    "/opt/adguardhome/scripts/healthcheck.sh"
               ],
               "Interval": 30000000000,
               "Timeout": 10000000000,
               "Retries": 3
          }

Image details:

adguardhome Dockerfile , adguardhome buildscript

HEALTHCHECK \
    --interval=30s \
    --timeout=10s \
    --retries=3 \
    CMD [ "/opt/adguardhome/scripts/healthcheck.sh" ]

[rhatdan]

Did you store it in Docker format? I am not sure HEALTHCHECK is stored in OCI Format?

[0xF4CED]

HEALTHCHECK is only a part of the Docker image spec and not OCI. The image in my example is in the Docker format.

12239 introduced support for health checks from image configurations. However, in certain cases, podman pull fails to parse or store the health check configuration.

I think the problem is easier to understand when following the steps in Steps to reproduce and Additional information, that I provided above.

[flouthoc]

@salevdns As far as I can see https://github.com/containers/podman/pull/12239 only corrected to get Imageconfig for images when it is in docker format, and I think it works in case of podman exec dinp docker save adguard/adguardhome | podman load cause docker save is saving in doceker format.

Is expectation here is to check if image metadata somehow contains Healthcheck if yes, include it in OCI's image's reserved field Healthcheck and podman must implement it while running the image ?

[0xF4CED]

I am not sure tbh. To me it looks like it should be stored in the docker format when pulling.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@flouthoc @iamkirkbater Any update on this?

[olifre]

In case it helps, I created a minimal test container in my accidentally reported duplicate issue:

• https://hub.docker.com/r/olifre/podman-healthcheck-test ( https://github.com/olifre/podman-healthcheck-test/blob/main/Dockerfile )

I observed the same problem pulling that one, and also the official iobroker container which is built from this Dockerfile. Pulling the same containers with Docker and running them there reveals a working healthcheck.

[FyiurAmron]

FWIW, I started getting this with Kaniko some time ago for no apparent reason. Older versions of Kaniko+podman had healthchecks working normally, and, as expected, Kaniko+docker works without any problems. Forced me to switch back to docker for the time being :/