docker-registry-proxy behind traefik

[rule88]

Hi, we try to get the docker-registry-proxy working behing Traefik. In this case Traefik is used to manage certificates and DNS entries, but somehow we have no success in getting the registry-proxy working properly. This is how it is suppose to work:

client --> traefik --> docker-registry-proxy --> src registry

But in Traefik we see messages like:

10.0.0.15 - - [22/Feb/2021:10:16:59 +0000] "CONNECT - HTTP/1.1" 404 19 "-" "-" 1725 "-" "-" 0ms

Can you elaborate on the caveats that might have been introduced with yet another proxy between the client and the docker-registry-proxy?

This is where we are right now using docker-compose (non-working):

version: "3"
services:
  traefik:
    image: traefik:v2.2
    restart: always
    command:
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--entrypoints.docker-proxy.address=:3128"
      - "--certificatesresolvers.letsencrypt.acme.tlschallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.email=support@domain.com"
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
    ports:
      - "3128:3128"
    volumes:
      - certs:/letsencrypt
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - proxy
    labels:
      # global redirect to https
      - "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.entrypoints=web"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
      # middleware redirect
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
      # Serve itself
      - "traefik.http.routers.traefik.rule=Host(`traefik.domain.com`)"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.tls.certresolver=letsencrypt"
      - "traefik.http.services.traefik.loadbalancer.server.port=8080"
      - "traefik.http.routers.traefik.middlewares=traefik-forward-auth"
  docker_proxy:
    image: rpardini/docker-registry-proxy:latest-debug
    restart: always
    labels:
      - "traefik.http.routers.proxy-docker-domain-com.rule=Host(`proxy.docker.domain.com`)"
      - "traefik.http.routers.proxy-docker-domain-com.entrypoints=docker-proxy"
      - "traefik.http.routers.proxy-docker-domain-com.service=proxy-docker-domain-com"
      - "traefik.http.services.proxy-docker-domain-com.loadbalancer.server.port=3128"
    environment:
      ENABLE_MANIFEST_CACHE: 'true'
      DEBUG: 'true'
    volumes:
      - docker_proxy_cache:/docker_mirror_cache
      - docker_proxy_certs:/ca
    networks:
      - proxy

[rpardini]

Hello. I have seen this (docker-registry-proxy) being used behind a simple TCP-based proxy like nginx. Since this (docker-registry-proxy) uses CONNECT proxies, Man-in-the-mIddle certificates, etc, it makes very little sense to try to expose via a TLS-terminating proxy like Traefik.

Can you elaborate on what you're trying to achieve? A super-fast data-leaking public proxy mirror?

[rule88]

Hi @rpardini thanks for looking into my request.

I am certainly not trying to provide the internet with a high-speed free mirror, the opposite is true, I want to keep it local.

I use Traefik (internally) to manage internal tools with their (sub)domains and routes. This way we have a single entry-point for all traffic. The function of Traefik would be as simple as forwarding traffic destined (port: 3128, host: proxy.docker.domain.com) to the docker proxy (service: proxy-docker-domain-com).

[rule88]

It btw thus isn't required to use the TLS-terminating abilities of Traefik in this case. Could you share the nginx config that you have seen earlier, that would be sufficient to dig further.

[rule88]

Hi @rpardini ,

I got it to work in combination with Traefik, so here is my config for future reference:

version: "3"
services:
  traefik:
    image: traefik:v2.2
    restart: always
    command:
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--entrypoints.docker-proxy.address=:3128"
      - "--certificatesresolvers.letsencrypt.acme.tlschallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.email=support@domain.com"
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
    ports:
      - "3128:3128"
    volumes:
      - certs:/letsencrypt
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - proxy
    labels:
      # global redirect to https
      - "traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`)"
      - "traefik.http.routers.http-catchall.entrypoints=web"
      - "traefik.http.routers.http-catchall.middlewares=redirect-to-https"
      # middleware redirect
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
      # Serve itself
      - "traefik.http.routers.traefik.rule=Host(`traefik.domain.com`)"
      - "traefik.http.routers.traefik.entrypoints=websecure"
      - "traefik.http.routers.traefik.tls.certresolver=letsencrypt"
      - "traefik.http.services.traefik.loadbalancer.server.port=8080"
      - "traefik.http.routers.traefik.middlewares=traefik-forward-auth"
  docker-proxy:
    image: rpardini/docker-registry-proxy:latest-debug
    restart: always
    labels:
      - "traefik.tcp.routers.docker-proxy.rule=HostSNI(`*`)"
      - "traefik.tcp.routers.docker-proxy.entrypoints=docker-proxy"
      - "traefik.tcp.routers.docker-proxy.service=docker-proxy"
      - "traefik.tcp.services.docker-proxy.loadbalancer.server.port=3128"
    environment:
      ENABLE_MANIFEST_CACHE: 'true'
      DEBUG: 'true'
    volumes:
      - docker_proxy_cache:/docker_mirror_cache
      - docker_proxy_certs:/ca
    networks:
      - proxy

There were 2 caveats, the first was to use TCP instead of HTTP routing in Traefik, the second was to use HostSNI with a wildcard: HostSNI(`*`). Documentation on the later can be found here

[tomaszkiewicz]

Maybe it's not a "drop in" solution, but I've got some issues with classic docker registry behind Traefik and adding these two options (in my case for helm chart) solved the issue:

      - "--entryPoints.web.proxyProtocol.insecure"
      - "--entryPoints.web.forwardedHeaders.insecure"