search

openebs / mayastor

Dynamically provision Stateful Persistent Replicated Cluster-wide Fabric Volumes & Filesystems for Kubernetes that is provisioned from an optimized NVME SPDK backend data storage stack.
Apache License 2.0
746 stars 107 forks source link

Small fix for release.sh so that it will work with podman's docker cli #1688

Closed ThorbenJ closed 3 months ago

ThorbenJ commented 4 months ago

Describe the bug Although podman pretends to provide a compatible docker cli, it's not 100% compatible. I made a small change to release.sh that allowed it to work, so that that I could build for arm64

To Reproduce Install podman instead of docker, and then run release.sh to build to container images

Expected behavior All images built and loaded

Screenshots

root@builder01:~/Git/openebs/mayastor-control-plane# ./scripts/release.sh --skip-publish
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
/nix/store/cc646hl0ba9rywbz7a4m7zihyr8zir2y-cargo-vendor-dir
Cargo vendored dependencies pre-fetched after 1 attempt(s)
Building openebs/mayastor-agent-core:v2.7.0 ...
/nix/store/2377gm3dwv2z7bmjb2cxs9sik27w4w7j-docker-image-mayastor-agent-core.tar.gz
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Error: payload does not match any of the supported image formats:
 * oci: initializing source oci:agents.core-image:: open agents.core-image/index.json: not a directory
 * oci-archive: loading index: open /var/tmp/oci1208288157/index.json: no such file or directory
 * docker-archive: loading tar component manifest.json: archive/tar: invalid tar header
 * dir: open agents.core-image/manifest.json: not a directory

OS info (please complete the following information):

Additional context It doesn't like gz tar files apparently, and I'm not sure about the lack of extention.

I changed

    $DOCKER load -i $archive-image

to this

    zcat $archive-image > $archive-image.tar
    $DOCKER load -i $archive-image.tar
    rm $archive-image.tar

(Lines before and after remain the same.

i.e. I needed gunzip the file, and (I believe) keep the .tar extention - and it now works fine. It generated and loaded all the images from both mayastor repos.

tiagolobocastro commented 4 months ago

Thanks for raising this @ThorbenJ Would you like to also raise a PR for this yourself?

Btw had a quick look and seems podman does support .tar.gz?

ThorbenJ commented 4 months ago

I am using Debian Stable my podman does not like it, its entirely possible that the latest now does; or that its the lack of explicit file extension.

I figured I'd share this for anyone else in my boat, as it was a simple compatibility fix.

FYI

 podman --version
podman version 4.3.1
tiagolobocastro commented 3 months ago

So I tried this on my box and seems to work, though my podman is more recent:

❯ podman load <$(nix-build -A images.mayastor-io-engine)                                                                                                                                                        ─╯
WARN[0000] Using cgroups-v1 which is deprecated in favor of cgroups-v2 with Podman v5 and will be removed in a future version. Set environment variable `PODMAN_IGNORE_CGROUPSV1_WARNING` to hide this warning. 
Getting image source signatures
Copying blob 252ebb56fa0a done   | 
Copying config 0b3387f3d1 done   | 
Writing manifest to image destination
Loaded image: localhost/openebs/mayastor-io-engine:5fa8b486b675

❯ podman --version                                                                                                                                                                                              ─╯
podman version 5.0.3
ThorbenJ commented 3 months ago

Ss Debian Stable appears to be a major version behind. This issue documents a work around for users of that version of podman; as such that might just be enough and therefore the issue could be closed.

tiagolobocastro commented 3 months ago

Also this seems to work btw:

DOCKER_HOST=unix:///var/run/user/1000/podman/podman.sock docker image ls
tiagolobocastro commented 3 months ago

I've raised #1703 on the dataplane repo, it checks if docker or podman are available, and makes use of it. Also I'll probably enhance this to simply use skopeo by default, afterall we don't really need to load the images in the host system if we just want to push them to a registry.

tiagolobocastro commented 3 months ago

Release script now can pick up docker or podman. If your podman version is too old you can:

  1. use --skopeo-copy to avoid using the host service, and copy directly to the registry.
  2. use podman from nix-shell: nix-shell -p podman --run "./scripts/release.sh --skip-publish"
  3. set DOCKER_HOST and use docker binary with podman service: DOCKER_HOST=unix:///var/run/user/1000/podman/podman.sock "./scripts/release.sh --skip-publish"
  4. simply run "./scripts/release.sh --skip-publish" and script will decompress the image if required
  5. explicitly use --tar "./scripts/release.sh --tar --skip-publish"
tiagolobocastro commented 3 months ago

A bug was introduced, fixing this...

tiagolobocastro commented 3 months ago

Alright, re-fixed and also now, if using podman and load of the tar.gz fails, we use zcat and load the tar instead. Also you can opt in from the start with --tar.