eth-cscs / sarus

OCI-compatible engine to deploy Linux containers on HPC environments.
https://sarus.readthedocs.io/en/stable/
BSD 3-Clause "New" or "Revised" License
127 stars 10 forks source link

Pull RepoDigest #1

Open kniec opened 4 years ago

kniec commented 4 years ago

Pulling an image based on a sha256 hash is not possible. I reckon it would benefit reproducibility to pull an image based on the RepoDigest:

$ docker inspect -f '{{.RepoDigests}}' qnib/uplain-osu-benchmark
[qnib/uplain-osu-benchmark@sha256:cd7957a4291dc0a6b34c4ae033c9b224fd2647fc1337459bfbf34f9ba046e396]

Docker allows to download an image based on the digest:

$ docker pull qnib/uplain-osu-benchmark@sha256:cd7957a4291dc0a6b34c4ae033c9b224fd2647fc1337459bfbf34f9ba046e396
sha256:cd7957a4291dc0a6b34c4ae033c9b224fd2647fc1337459bfbf34f9ba046e396: Pulling from qnib/uplain-osu-benchmark
Digest: sha256:cd7957a4291dc0a6b34c4ae033c9b224fd2647fc1337459bfbf34f9ba046e396
Status: Image is up to date for qnib/uplain-osu-benchmark@sha256:cd7957a4291dc0a6b34c4ae033c9b224fd2647fc1337459bfbf34f9ba046e396

Sarus is breaking when using a digest:

$ sarus pull qnib/uplain-osu-benchmark@sha256:cd7957a4291dc0a6b34c4ae033c9b224fd2647fc1337459bfbf34f9ba046e396
# image            : index.docker.io/qnib/uplain-osu-benchmark@sha256:cd7957a4291dc0a6b34c4ae033c9b224fd2647fc1337459bfbf34f9ba046e396
# cache directory  : "/home/ubuntu/.sarus/cache"
# temp directory   : "/tmp"
# images directory : "/home/ubuntu/.sarus/images"
Failed authentication for image 'index.docker.io/qnib/uplain-osu-benchmark@sha256:cd7957a4291dc0a6b34c4ae033c9b224fd2647fc1337459bfbf34f9ba046e396'
Did you perform a login with the proper credentials?
See 'sarus help pull' (--login option)
vsoch commented 3 years ago

Just want to chime in here that I would really like this feature too! I just was really sad to realize that I can't pull a specific digest with sarus:

sarus pull python@sha256:02311d686cd35b0f838854d6035c679acde2767a4fd09904e65355fbd9780f8a

# image            : index.docker.io/library/python@sha256:02311d686cd35b0f838854d6035c679acde2767a4fd09904e65355fbd9780f8a
# cache directory  : "/home/vanessa/.sarus/cache"
# temp directory   : "/tmp"
# images directory : "/home/vanessa/.sarus/images"
Failed to pull image 'index.docker.io/library/python@sha256:02311d686cd35b0f838854d6035c679acde2767a4fd09904e65355fbd9780f8a'
The image is not present in the remote registry.

Is it the case that it doesn't exist? It is not - the image is pullable with docker:

docker pull python@sha256:02311d686cd35b0f838854d6035c679acde2767a4fd09904e65355fbd9780f8a

docker.io/library/python@sha256:02311d686cd35b0f838854d6035c679acde2767a4fd09904e65355fbd9780f8a: Pulling from library/python
540db60ca938: Pull complete 
d037ddac5dde: Pull complete 
629719f9106a: Pull complete 
f9ef3a05a91e: Pull complete 
0faf4e7f2207: Pull complete 
Digest: sha256:02311d686cd35b0f838854d6035c679acde2767a4fd09904e65355fbd9780f8a
Status: Downloaded newer image for python@sha256:02311d686cd35b0f838854d6035c679acde2767a4fd09904e65355fbd9780f8a
docker.io/library/python@sha256:02311d686cd35b0f838854d6035c679acde2767a4fd09904e65355fbd9780f8a

I'd like to add sarus to Singularity Registry HPC which will make sarus containers available via environment modules, but it will definitely fall short if I can't pull a specific digest.

Madeeks commented 3 years ago

Hi @vsoch, thanks for reporting this behavior. The error message is definitely misleading, likely due to the fact that Sarus currently cannot handle specific digest strings/identifiers.

There are plans for an overall update of the image pulling/management code, with the intent to add a number of user-requested features (pulling by digest is definitely one of these!) and overcome several limitations of the current implementation.

While it will take us a little longer to get there, in the short term we can look into making the error message less ambiguous when using digests.