containers / skopeo

Work with remote images registries - retrieving information, images, signing content
Apache License 2.0
7.75k stars 756 forks source link

Sync to private registry fails while copying is working #2350

Open alexanderbazhenoff opened 3 weeks ago

alexanderbazhenoff commented 3 weeks ago

Hi,

For example:

skopeo copy --all docker://alpine docker://registry.domain/alpine/alpine --dest-tls-verify=false \
    --dest-creds user:password

is working (all architectures with latest tag was copied) while:

skopeo sync --all --src docker --dest docker alpine registry.domain/alpine \
    --dest-tls-verify=false --dest-creds user:password

was failed:

INFO[0000] Tag presence check                            imagename=alpine tagged=false
INFO[0000] Getting tags                                  image=docker.io/library/alpine
INFO[0001] Copying image ref 1/161                       from="docker://alpine:2.6" to="docker://registry.domain/alpine/alpine:2.6"
Getting image source signatures
Copying blob 2a3ebcb7fbcc done  
Writing manifest to image destination
FATA[0037] Error copying ref "docker://alpine:2.6": Error writing manifest "{\"name\":\"alpine/alpine\",\"tag\":\"2.6\",\"architecture\":\"amd64\",\"fsLayers\":[{\"blobSum\":\"sha256:2a3ebcb7fbcc29bf40c4f62863008bb573acdea963454834d9483b3e5300c45d\"}],\"history\":[{\"v1Compatibility\":\"{\\\"id\\\":\\\"dd807873c9a21bcc82e30317c283e6601d7e19f5cf7867eec34cdd1aeb3f099e\\\",\\\"created\\\":\\\"2016-01-18T18:32:39.162138276Z\\\",\\\"container\\\":\\\"556a728876db7b0e621adc029c87c649d32520804f8f15defd67bb070dc1a88d\\\",\\\"container_config\\\":{\\\"Hostname\\\":\\\"556a728876db\\\",\\\"Domainname\\\":\\\"\\\",\\\"User\\\":\\\"\\\",\\\"AttachStdin\\\":false,\\\"AttachStdout\\\":false,\\\"AttachStderr\\\":false,\\\"Tty\\\":false,\\\"OpenStdin\\\":false,\\\"StdinOnce\\\":false,\\\"Env\\\":null,\\\"Cmd\\\":[\\\"/bin/sh\\\",\\\"-c\\\",\\\"#(nop) ADD file:7dee8a455bcc39013aa168d27ece9227aad155adbaacbd153d94ca60113f59fc in /\\\"],\\\"Image\\\":\\\"\\\",\\\"Volumes\\\":null,\\\"WorkingDir\\\":\\\"\\\",\\\"Entrypoint\\\":null,\\\"OnBuild\\\":null,\\\"Labels\\\":null},\\\"docker_version\\\":\\\"1.8.3\\\",\\\"config\\\":{\\\"Hostname\\\":\\\"556a728876db\\\",\\\"Domainname\\\":\\\"\\\",\\\"User\\\":\\\"\\\",\\\"AttachStdin\\\":false,\\\"AttachStdout\\\":false,\\\"AttachStderr\\\":false,\\\"Tty\\\":false,\\\"OpenStdin\\\":false,\\\"StdinOnce\\\":false,\\\"Env\\\":null,\\\"Cmd\\\":null,\\\"Image\\\":\\\"\\\",\\\"Volumes\\\":null,\\\"WorkingDir\\\":\\\"\\\",\\\"Entrypoint\\\":null,\\\"OnBuild\\\":null,\\\"Labels\\\":null},\\\"architecture\\\":\\\"amd64\\\",\\\"os\\\":\\\"linux\\\",\\\"Size\\\":4501436}\"}],\"schemaVersion\":1,\"signatures\":[{\"header\":{\"jwk\":{\"crv\":\"P-256\",\"kid\":\"6ZYY:RZG3:TGDG:RDUC:QIVO:NLJP:JRMK:G756:V4RG:2PJI:HAAK:KU3L\",\"kty\":\"EC\",\"x\":\"nb9JsXM7gIjWGS_ooJTh29fB2d6Wt1wspibNok-0Z-s\",\"y\":\"E2M4bgCH3nhfaCk7u01_9WTNoqqJGq1GzvUbFlEdyu0\"},\"alg\":\"ES256\"},\"signature\":\"nweRr0d6eO_JhR3iO88YpDBFb7742VsCyPF-k4UyQh-TPyHhYE3ETyA9aGGGONGaUUeurSyXGBUz3ydV_fSgEA\",\"protected\":\"eyJmb3JtYXRMZW5ndGgiOjEyODMsImZvcm1hdFRhaWwiOiJmUSIsInRpbWUiOiIyMDI0LTA2LTA2VDEwOjQ3OjEzWiJ9\"}]}": Error uploading manifest 2.6 to registry.domain/alpine/alpine: received unexpected HTTP status: 500 Internal Server Error 

(full log with --debug flag is on pastebin)

Any attempt of cloning was failed (apline, busybox, ubuntu).

Registry and registry-UI settings are:

---
version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/docker-registry
  delete:
    enabled: true
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
    Access-Control-Allow-Origin: ['http://localhost']
    Access-Control-Allow-Methods: ['HEAD', 'GET', 'OPTIONS', 'DELETE']
    Access-Control-Allow-Headers: ['Authorization', 'Accept', 'Cache-Control']
    Access-Control-Max-Age: [1728000]
    Access-Control-Allow-Credentials: [true]
    Access-Control-Expose-Headers: ['Docker-Content-Digest']
auth:
  htpasswd:
    realm: basic-realm
    path: /etc/docker/registry/auth/passwdfile
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

What am I doing wrong?

Thanks for any info in advance.

mtrmac commented 3 weeks ago

Thanks for reaching out.

“Internal server error” really must be diagnosed server-side. What do the server logs say?

At a guess, some versions of some registry implementations reject schema1 images by default. (You are listing no version numbers, so I couldn’t investigate anyway.) But that wouldn’t explain how skopeo copy works, so either it’s not that, or there is some variable not accounted for in the above.

(Also, consider using a newer version of Skopeo; this one seems to be at least three years old.)