search

Trow-Registry / trow

Container Registry and Image Management for Kubernetes Clusters
https://trow.io
Apache License 2.0
912 stars 102 forks source link

Support Proxying/Mirroring #77

Open amouat opened 4 years ago

amouat commented 4 years ago

The main use case for trow is running inside a cluster. Therefore it will often work alongside other registries, such as a public registry like Docker Hub & Quay or an enterprise wide registry (often artifactory or Nexus). It's essential that trow is able to transparently and easily mirror or proxy images from these clusters.

There is some interesting discussion on this topic here: https://github.com/opencontainers/distribution-spec/issues/12

golyalpha commented 2 years ago

Although mirroring seems to be functional at this point, there are commonly encountered edge cases where it causes a problem.

For example, pulling a multi-platform image through Trow on one architecture, and then attempting to pull it again on another architecture, causes Trow to serve up the image for the former (now incorrect) architecture, rather than the manifest referencing all the supported architectures and subsequently proxying the download for the correct architecture.

Example:

  1. Dev runs docker pull registry.local/f/docker/nginx, pulling down the linux/amd64 version of the image 2a. Dev deploys app to arm64 K8s cluster 2b. Admission check passes because registry.local/f/docker/nginx is now available on the registry
  2. K8s pulls in the linux/amd64 image, unaware that there is actually a multi-platform manifest under that tag, rather than an image

This sort of highlights another issue - admission check fails for proxied images when they aren't pre-pulled in Trow. This may actually be a feature, allowing a cluster administrator to control which images from Docker Hub may be used on the cluster, by having to pre-pull them into Trow, however, there should at least be an option to make Trow check availability of proxied images in upstream registries before the admission check fails.

amouat commented 2 years ago

Good catch. I'll have to think about the right solution here.

golyalpha commented 2 years ago

So, I just found out through further testing that I was completely wrong - trow completely ignores what platform was requested and always requests linux/amd64.

It's literally impossible to get another platform. I even tested with explicitly specifying the linux/arm64 platform, and got this:

$ docker pull registry.local/f/docker/alpine --platform linux/arm64
Using default tag: latest
latest: Pulling from f/docker/alpine
59bf1c3509f3: Already exists 
Digest: sha256:e7d88de73db3d3fd9b2d63aa7f447a10fd0220b7cbf39803c803f2af9ba256b3
Status: Downloaded newer image for registry.local/f/docker/alpine:latest
WARNING: image with reference registry.local/f/docker/alpine was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64
registry.local/f/docker/alpine:latest
amouat commented 2 years ago

Yeah, that makes sense. I'll need to look into it, but I suspect I need to pull the images for all architectures.

golyalpha commented 2 years ago

I mean, idk how Docker's own registry does it - but it doesn't have this issue when it comes to pulling images through it in Docker Hub Proxy mode.