Default compression format for push is zstd:chunked after Podman machine v5.1

tnk4on commented 5 months ago

Issue Description

I don't know if this is a bug or a spec since v5.1, but I noticed it when testing push. When I do a podman push, it uses zstd:chunked instead of gzip. In v5.0, it continues to be gzip.

The default value for c/common seems to be gzip. https://github.com/containers/common/blob/b7856429642a2d2eb5b3cc6656b7057ecfc4b4ad/pkg/config/default.go#L358

Steps to reproduce the issue

Podman machine v5.1

% podman machine init --now
% podman machine ssh
core@localhost:~$ podman run -d -p 5000:5000 --name registry docker.io/library/registry
core@localhost:~$ cat > Containerfile <<EOF
FROM alpine
RUN arch |tee arch.txt
EOF
core@localhost:~$ podman build -t zstd-test .
core@localhost:~$ podman push zstd-test localhost:5000/zstd-test --tls-verify=false
core@localhost:~$ skopeo inspect docker://localhost:5000/zstd-test --tls-verify=false
{
    "Name": "localhost:5000/zstd-test",
    "Digest": "sha256:9b68526d8e4d288aa0ed84cb8ce48792745e2cf2139be06b94f39b6f6fc1fa54",
    "RepoTags": [
        "latest"
    ],
    "Created": "2024-05-27T01:44:56.438062327Z",
    "DockerVersion": "",
    "Labels": {
        "io.buildah.version": "1.36.0-dev"
    },
    "Architecture": "arm64",
    "Os": "linux",
    "Layers": [
        "sha256:0ddf62188dee1d164be56de334cb2bc1acfd782272153e715536a09bbbe902e4",
        "sha256:9a7d95119886a69c32eb66b115e1a01546c9628c7910098eea4d01d93a3c6c95"
    ],
    "LayersData": [
        {
            "MIMEType": "application/vnd.oci.image.layer.v1.tar+zstd",
            "Digest": "sha256:0ddf62188dee1d164be56de334cb2bc1acfd782272153e715536a09bbbe902e4",
            "Size": 4056584,
            "Annotations": {
                "io.github.containers.zstd-chunked.manifest-checksum": "sha256:4e71d547fe3635193dafbc8670a47b7e9d480692e79cf55f5064af3f7de45d2c",
                "io.github.containers.zstd-chunked.manifest-position": "4022695:17567:142475:1",
                "io.github.containers.zstd-chunked.tarsplit-checksum": "sha256:c65ec54cd3f97904118e0ad57583964ee357e3ca1f8931c4cc21a3d8bac3789f",
                "io.github.containers.zstd-chunked.tarsplit-position": "4040270:16242:444602"
            }
        },
        {
            "MIMEType": "application/vnd.oci.image.layer.v1.tar+zstd",
            "Digest": "sha256:9a7d95119886a69c32eb66b115e1a01546c9628c7910098eea4d01d93a3c6c95",
            "Size": 940,
            "Annotations": {
                "io.github.containers.zstd-chunked.manifest-checksum": "sha256:d0fd86c02dc0c621df12b93e3a53385dc4f285779ed236dbd93c51eea92d2ec9",
                "io.github.containers.zstd-chunked.manifest-position": "245:262:760:1",
                "io.github.containers.zstd-chunked.tarsplit-checksum": "sha256:979b234d4e938cb1168dd2ae47897efe880585b4772405dcaea33d12efc9ea54",
                "io.github.containers.zstd-chunked.tarsplit-position": "515:353:5207"
            }
        }
    ],
    "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ]
}

Podman machine v5.0

$ skopeo inspect docker://localhost:5000/zstd-test --tls-verify=false
{
    "Name": "localhost:5000/zstd-test",
    "Digest": "sha256:8f14612bcccaa141f0fd0670802c6d4e7aa97cd0c49e33bae7bfea09d076279b",
    "RepoTags": [
        "latest"
    ],
    "Created": "2024-05-27T01:33:09.119470301Z",
    "DockerVersion": "",
    "Labels": {
        "io.buildah.version": "1.35.4"
    },
    "Architecture": "arm64",
    "Os": "linux",
    "Layers": [
        "sha256:65dd4daf043869a7d35276bac0bb659e66483e9f2dcf749adae5563f0268f17e",
        "sha256:317164bf9b9294bdbb87177c4a5e88c3b4f4f3bbaa60b068c06a87caa3d3b4b4"
    ],
    "LayersData": [
        {
            "MIMEType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "Digest": "sha256:65dd4daf043869a7d35276bac0bb659e66483e9f2dcf749adae5563f0268f17e",
            "Size": 4198001,
            "Annotations": null
        },
        {
            "MIMEType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "Digest": "sha256:317164bf9b9294bdbb87177c4a5e88c3b4f4f3bbaa60b068c06a87caa3d3b4b4",
            "Size": 193,
            "Annotations": null
        }
    ],
    "Env": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ]
}

Describe the results you received

"MIMEType": "application/vnd.oci.image.layer.v1.tar+zstd",

Describe the results you expected

"MIMEType": "application/vnd.oci.image.layer.v1.tar+gzip",

podman info output

% podman info
host:
  arch: arm64
  buildahVersion: 1.36.0-dev
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.10-1.fc40.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: '
  cpuUtilization:
    idlePercent: 99.46
    systemPercent: 0.27
    userPercent: 0.27
  cpus: 5
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "40"
  eventLogger: journald
  freeLocks: 2046
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 6.8.8-300.fc40.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 1410076672
  memTotal: 2045304832
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.10.0-1.20240506173313423293.main.51.g069ab45.fc40.aarch64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.11.0-dev
    package: netavark-1.10.1-1.20240513124445753694.main.112.gd982b8b.fc40.aarch64
    path: /usr/libexec/podman/netavark
    version: netavark 1.11.0-dev
  ociRuntime:
    name: crun
    package: crun-1.14.4-1.20240424212458225367.main.39.gd075e53.fc40.aarch64
    path: /usr/bin/crun
    version: |-
      crun version UNKNOWN
      commit: 320753b75c4e30085176ffc515936df286edbde2
      rundir: /run/user/501/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240426.gd03c4e2-1.fc40.aarch64
    version: |
      pasta 0^20240426.gd03c4e2-1.fc40.aarch64-pasta
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-2.fc40.aarch64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 0h 16m 44.00s
  variant: v8
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphRootAllocated: 106769133568
  graphRootUsed: 4285399040
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 3
  runRoot: /run/user/501/containers
  transientStore: false
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 5.1.0-dev-c9808e7ed
  Built: 1715558400
  BuiltTime: Mon May 13 09:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.2
  Os: linux
  OsArch: linux/arm64
  Version: 5.1.0-dev-c9808e7ed

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

% sw_vers
ProductName:        macOS
ProductVersion:     14.5
BuildVersion:       23F79

% podman version
Client:       Podman Engine
Version:      5.1.0-dev
API Version:  5.1.0-dev
Go Version:   go1.22.2
Git Commit:   3718c197c90f497b805039cd8efad55417ce5fdc
Built:        Thu Apr 18 10:39:38 2024
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      5.1.0-dev-c9808e7ed
API Version:  5.1.0-dev-c9808e7ed
Go Version:   go1.22.2
Built:        Mon May 13 09:00:00 2024
OS/Arch:      linux/arm64

Additional information

No response

tnk4on commented 5 months ago

The cause is that the setting of /usr/share/containers/containers.conf in machine-os has been changed.

core@localhost:~$ sudo grep compression_format /usr/share/containers/containers.conf
#compression_format = "gzip"
compression_format = "zstd:chunked"

rhatdan commented 5 months ago

The containers-conf changes should only be in Rawhide not a released version yet.

rhatdan commented 5 months ago

@tnk4on Did you find any issues with zstd:chunked, or just noticed the change leaked in.

tnk4on commented 5 months ago

@rhatdan Yes, there is nothing wrong with zstd:chunked itself. I just noticed a behavior that is not in the documentation or in the release.

github-actions[bot] commented 4 months ago

A friendly reminder that this issue had no activity for 30 days.

Luap99 commented 2 months ago

I think that should be fixed in the latest 5.2 images

containers / podman