Running Behind NGINX on Non-Standard Port

[steenbag]

Hi, I've been trying to use the behind NGINX example, and I have everything more or less up and running. I am however running into an issue, I have another web server running on my VPS, which consumes ports 443 and 80. As such, I cannot run the NGINX example on the standard ports. Instead I'm using 8000, which forwards to 443 on the nginx container.

I also changed the port for the registry it's self so that I could expose port 5000 via nginx and get an https proxy for remote connections.

The registry works fine, I can login, push and pull. Everything works in the UI (i.e. I can login, manage users, etc) except that whenever the app redirects, it loses the port number in the URL. As a result, I keep getting thrown back into my other web app that is running on ports 80/443.

For example, when I come to the app without a session (https://registry.mydomain.com:8000) it detects I have no session and throws me to https://registry.mydomain.com/login/auth (which gives me a 404 from my other app). I have to manually add :8000 back in to get to the login page. Several other pages in the app have the same result (user creation, user update, etc).

Is there some configuration change I can make to fix the redirects? I've attached my configuration below:

Compose file:

version: '2'
services:
  nginx:
    image: nginx:1.10
    ports:
      - 8000:443
      - 5000:443
    volumes:
      - ./conf/nginx:/etc/nginx/conf.d/:ro
      - /opt/jenkins:/certs/:ro
      - ./log/nginx/:/var/log/nginx/
    depends_on:
      - registry
      - registry-web
  registry-web:
    image: hyper/docker-registry-web:latest
    ports:
      - 127.0.0.1:8080:8080
    volumes:
       - ./conf/registry-web:/conf:ro
       - ./db:/data
       - /opt/jenkins:/certs/:ro
    depends_on:
       - registry
  registry:
    image: registry:2.4.1
    ports:
      - 127.0.0.1:5050:5050
    volumes:
      - ./conf/registry:/etc/docker/registry:ro
      - ./data:/var/lib/registry
      - /opt/jenkins:/certs/:ro

conf/nginf/default.conf:

upstream docker-registry {
    server registry:5050;
}

upstream docker-registry-web {
    server registry-web:8080;
}

server {
    server_name registry.***.com;
    listen 80;

#    *** for https support uncomment following lines ***
    listen 443 ssl;

    # SSL keys
    ssl_certificate     /certs/comodo-bundle.crt;
    ssl_certificate_key /certs/cert.key;

    # Recommendations from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;

    # disable any limits to avoid HTTP 413 for large image uploads
    client_max_body_size 0;

    # required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
    chunked_transfer_encoding on;

    location /v2/ {
      # Do not allow connections from docker 1.5 and earlier
      # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
      if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*\$" ) {
        return 404;
      }

      proxy_pass                          http://docker-registry;
      proxy_set_header  Host              $http_host;   # required for docker client's sake
      proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
      proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header  X-Forwarded-Proto $scheme;
      proxy_read_timeout                  900;
    }

    location / {
      proxy_pass                          http://docker-registry-web;
      proxy_set_header  Host              $http_host;
      proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
      proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header  X-Forwarded-Proto $scheme;
    }
}

conf/registry/config.yml:

version: 0.1
log:
  level: info
  fields:
    service: registry

storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
  delete:
    enabled: true

http:
  addr: 0.0.0.0:5050

auth:
  token:
    realm: XXX
    service: XXX
    issuer: test
    rootcertbundle: /certs/comodo-bundle.crt

notifications:
  endpoints:
    - name: listener
      url: http://registry-web:8080/api/notification
      timeout: 500ms
      threshold: 5
      backoff: 1s

health:
  storagedriver:
    enabled: true
    interval: 10s

threshold: 3

conf/registry-web/config.yml:

registry:
   url: http://registry:5050/v2
   name: https://registry.***.com:5000
   readonly: false
   auth:
     enabled: true
     key: /certs/cert.key
     issuer: test

[mkuchin]

Thanks for question, @steenbag. There is no easy way to run it on a non-standard port currently. However, if you can use nginx to reverse proxy both your web app and registry ui and run registry ui on non-standard context path:

# web registry context path
# empty string for root context, /app to make web registry accessible on http://host/app
context_path: ''

Will this config option be able to solve your issue?

[steenbag]

I wasn't able to use the context path since the other web app (jenkins) was listening on all port 443 requests. What I ended up doing (which seems to be successful so far) is move the nginx proxy in front of both apps. I also migrated my jenkins install to a docker container as well, so now I have one NGINX container acting as an SSL terminator, and the reverse proxy to the individual containers as appropriate based on the domain name.

Thanks, Andrew