uber-archive / makisu

Fast and flexible Docker image building tool, works in unprivileged containerized environments like Mesos and Kubernetes.
Apache License 2.0
2.41k stars 154 forks source link

Base image pull by digest doesn't work #235

Closed hatemosphere closed 4 years ago

hatemosphere commented 5 years ago

Apparently, makisu doesn't work with format like this (tag + digest):

FROM gcr.io/makisu-project/makisu-alpine:v0.1.11@sha256:fc80186c3b46b19d3fc2df062250a85c95dd57380ac4fe19ad20d26358b9a3a5

or even like this (digest only):

FROM gcr.io/makisu-project/makisu-alpine@sha256:fc80186c3b46b19d3fc2df062250a85c95dd57380ac4fe19ad20d26358b9a3a5

I think it's crucial to be able to pull base images via digest since this way is defined as a best practice and supported by at least Docker and Kubernetes.

AurelienGasser commented 4 years ago

@yiranwang52 The fix doesn't seem to work for DockerHub images.

With

FROM python:3.6-buster@sha256:6655d57ae9272f09839a25d0ec0d272513663d99a4ad7af8b061c83a76d10295

I get:

{"level":"info","ts":1592920823.65021,"msg":"* Step 1/2 (commit,modifyfs) : FROM python:3.6-buster@sha256:6655d57ae9272f09839a25d0ec0d272513663d99a4ad7af8b061c83a76d10295  (eaa36195)"}
{"level":"info","ts":1592920823.650249,"msg":"* Started pulling image index.docker.io/library/python:sha256:6655d57ae9272f09839a25d0ec0d272513663d99a4ad7af8b061c83a76d10295"}
{"level":"info","ts":1592920824.1015887,"msg":"* Removed 8 directories under /","duration":0}
{"level":"error","ts":1592920824.1018,"msg":"failed to execute build plan: execute stage: build stage 0: build node: do execute: execute step: get manifest: pull image index.docker.io/library/python@sha256:6655d57ae9272f09839a25d0ec0d272513663d99a4ad7af8b061c83a76d10295: pull manifest: unmarshal distribution manifest: unsupported manifest mediatype: application/vnd.docker.distribution.manifest.list.v2+json"}
TeoZosa commented 3 years ago

@yiranwang52 The fix doesn't seem to work for DockerHub images.

With

FROM python:3.6-buster@sha256:6655d57ae9272f09839a25d0ec0d272513663d99a4ad7af8b061c83a76d10295

I get:

{"level":"info","ts":1592920823.65021,"msg":"* Step 1/2 (commit,modifyfs) : FROM python:3.6-buster@sha256:6655d57ae9272f09839a25d0ec0d272513663d99a4ad7af8b061c83a76d10295  (eaa36195)"}
{"level":"info","ts":1592920823.650249,"msg":"* Started pulling image index.docker.io/library/python:sha256:6655d57ae9272f09839a25d0ec0d272513663d99a4ad7af8b061c83a76d10295"}
{"level":"info","ts":1592920824.1015887,"msg":"* Removed 8 directories under /","duration":0}
{"level":"error","ts":1592920824.1018,"msg":"failed to execute build plan: execute stage: build stage 0: build node: do execute: execute step: get manifest: pull image index.docker.io/library/python@sha256:6655d57ae9272f09839a25d0ec0d272513663d99a4ad7af8b061c83a76d10295: pull manifest: unmarshal distribution manifest: unsupported manifest mediatype: application/vnd.docker.distribution.manifest.list.v2+json"}

After spending a day on this same problem, I think I've found the solution.

Explanation:

The error: unsupported manifest mediatype: application/vnd.docker.distribution.manifest.list.v2+json

Means the image is multi-platform and you are using the digest that points to the image's Manifest List (the list of discrete platform-specific manifests) as opposed to a specific Image Manifest.

(Potential) Problem Source:

Obtaining the digest from the docker CLI (i.e., docker pull or docker images --digests):

ManifestList_Digest_CLI_Images ManifestList_Digest_CLI_Pull

Solution:

Use the digest of a platform-specific manifest, which you can get via docker manifest inspect <IMAGE> or via DockerHub:

Manifest_Digest_CLI Manifest_Digest_DockerHub

Makisu Build Output:

ManifestList_vs_Manifest_BuildOutput