HTTP Auth not supported

[mgreenwald-pm]

Hi,

We use an nginx container for authentication to our docker registry. As of now there isn't anyway to authenticate with this service before pulling images. It'd be nice if there was similar support for http_auth as there is for docker.io.

Maybe http_auth_user and http_auth_pass as environment variables?

Thanks

[mbentley]

In order to have it basically do a docker login to the registry where it uses HTTP instead of HTTPS, that is all on the engine configuration. The Engine that you're running migrator from would need the --insecure-registry <registry-fqdn-or-ip> parameter set. That'd allow the login to work to a HTTP endpoint.

[mgreenwald-pm]

@mbentley Even with --insecure-registry I am still getting:

[ERROR] curl => API failure1 [ERROR] Migration from v1 to v2 failed!

[mbentley]

~Are you using -e USE_INSECURE_CURL=true in your docker run command? That is needed for the curl commands to ignore a certificate that isn't trusted by default.~ Sorry just re-read the error. The full command you're running would be useful minus any credentials. Also what is the output from before the error so I can see what API call it would be making? I can then get a replication of what it is actually attempting to do for further troubleshooting.

[mgreenwald-pm]

Thanks for the help: docker run -it -v ~/.aws:/root/.aws:ro -v /var/run/docker.sock:/var/run/docker.sock -e V1_REGISTRY=registry.domain.com: -e V2_REGISTRY=registry.domain.com -e NO_LOGIN=true -e USE_INSECURE_CURL=true -e USE_HTTP=true docker/migrator 2>&1 | tee migration.log

It is attempting to Getting a list of images before the error.

Our registry runs in this order: AWS ELB -> CoreOS machine -> Nginx container -> Registry v1 container

[mbentley]

What sort of authentication is being done on the v1 registry (V1_REGISTRY) w/nginx? You have -e NO_LOGIN=true so if nginx is taking care of auth, the curl commands aren't going to pass any credentials.

The first curl command that we hit is:

curl ${V1_OPTIONS} -sf ${V1_PROTO}://${AUTH_CREDS}@${V1_REGISTRY}/v1/search?q= | jq -r '.results | .[] | .name'

Based off of the parameters you're passing, this should return data:

curl -v http://registry.domain.com/v1/search?q=

[mgreenwald-pm]

We are using auth_basic and htpasswd on nginx. The following output has NO_LOGIN removed.

[!!] Please login to registry.domain.com:
Username (flastname): 
WARNING: login credentials saved in /root/.dockercfg.
Login Succeeded

[INFO] Getting a list of images from registry.domain.com

[ERROR] curl => API failure

[mbentley]

If you're using basic auth, you should not be using -e NO_LOGIN=true. Did your command change when you created the above output? The code should never get to the docker login section if you have -e NO_LOGIN=true. From the output, the curl command I posted above would be the curl command it is running.

[mgreenwald-pm]

Yes, I removed NO_LOGIN.

[mbentley]

Whew, ok had me looking for a hole in the code 😄

[mgreenwald-pm]

But I am still getting the same errors regardless of how I attempt to use the migrator.

[mbentley]

Not sure if you missed https://github.com/docker/migrator/issues/109#issuecomment-303496589 but could you try to manually run the curl that is failing?

curl -v http://registry.domain.com/v1/search?q=

[mgreenwald-pm]

$ curl -u flastname -v https://registry.domain.com/v1/search?q=
Enter host password for user 'flastname':
*   Trying 1.1.1.1...
* Connected to registry.domain.com (1.1.1.1) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 692 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
*    server certificate verification OK
*    server certificate status verification SKIPPED
*    common name: registry.domain.com (matched)
*    server certificate expiration date OK
*    server certificate activation date OK
*    certificate public key: RSA
*    certificate version: #3
*    subject: CN=registry.domain.com
*    start date: Tue, 18 Apr 2017 00:00:00 GMT
*    expire date: Fri, 18 May 2018 12:00:00 GMT
*    issuer: C=US,O=Registrar,OU=Server CA 1B,CN=Registrar
*    compression: NULL
* ALPN, server did not agree to a protocol
* Server auth using Basic with user 'flastname'
> GET /v1/search?q= HTTP/1.1
> Host: registry.domain.com
> Authorization: Basic 1JS7gnJF5OdiITexBLUbuNSZVgJ=
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 404 NOT FOUND
< Content-Type: text/html
< Date: Tue, 23 May 2017 19:22:04 GMT
< Server: nginx/1.11.13
< Content-Length: 233
< Connection: keep-alive
< 
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>
* Connection #0 to host registry.domain.com left intact

[mbentley]

Make sure that you're not running -e USE_HTTP=true since according to the output, nginx is running SSL.

And it looks like either your v1 registry is not running search or nginx is not exposing /v1/search

[mgreenwald-pm]

I removed -e USE_HTTP=true as well.

Everything behind ../ is exposed once authenticated. So I guess I need to enable search on my registry.

[mgreenwald-pm]

Apparently search is part of the Docker v1 API so its not something that can be enabled or disabled.

[mbentley]

https://github.com/docker/docker-registry/blob/master/README.md#search-engine-options

Search-engine options

The Docker Registry can optionally index repository information in a database for the GET /v1/search endpoint. You can configure the backend with a configuration like:

The search_backend setting selects the search backend to use. If search_backend is empty, no index is built, and the search endpoint always returns empty results.

search_backend: The name of the search backend engine to use. Currently supported backends are: sqlalchemy If search_backend is neither empty nor one of the supported backends, it should point to a module.

[mgreenwald-pm]

Its configured to use sqlalchemy. :-/

[mbentley]

Let me do some testing when I get back to my computer and I'll see what I can find out. I have a similar setup with nginx.

[mbentley]

Hmm, there is something wrong with the v1 registry then.

$ curl -u demo -v https://registry.core.dckrindy.io/v1/search?q=
Enter host password for user 'demo':
*   Trying 192.168.1.50...
* TCP_NODELAY set
* Connected to registry.core.dckrindy.io (192.168.1.50) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate: core.dckrindy.io
* Server certificate: Let's Encrypt Authority X3
* Server certificate: DST Root CA X3
* Server auth using Basic with user 'demo'
> GET /v1/search?q= HTTP/1.1
> Host: registry.core.dckrindy.io
> Authorization: Basic ZGVtbzpkb2NrZxIxMjM=
> User-Agent: curl/7.51.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Wed, 24 May 2017 11:52:35 GMT
< Content-Type: application/json
< Content-Length: 346
< Connection: keep-alive
< Expires: -1
< Pragma: no-cache
< Cache-Control: no-cache
<
* Curl_http_done: called premature == 0
* Connection #0 to host registry.core.dckrindy.io left intact
{"num_results": 6, "query": "", "results": [{"description": null, "name": "mbentley/hello"}, {"description": null, "na
me": "mbentley/testssl"}, {"description": null, "name": "demo/docker-demo"}, {"description": null, "name": "library/de
bian"}, {"description": null, "name": "library/nginx"}, {"description": null, "name": "library/hello-world"}]}

or if I bypass nginx:

$ curl -v http://192.168.1.61:5000/v1/search?q=
*   Trying 192.168.1.61...
* TCP_NODELAY set
* Connected to 192.168.1.61 (192.168.1.61) port 5000 (#0)
> GET /v1/search?q= HTTP/1.1
> Host: 192.168.1.61:5000
> User-Agent: curl/7.51.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: gunicorn/19.1.1
< Date: Wed, 24 May 2017 11:54:55 GMT
< Connection: keep-alive
< Expires: -1
< Content-Type: application/json
< Pragma: no-cache
< Cache-Control: no-cache
< Content-Length: 346
<
* Curl_http_done: called premature == 0
* Connection #0 to host 192.168.1.61 left intact
{"num_results": 6, "query": "", "results": [{"description": null, "name": "mbentley/hello"}, {"description": null, "name": "mbentley/testssl"}, {"description": null, "name": "demo/docker-demo"}, {"description": null, "name": "library/de
bian"}, {"description": null, "name": "library/nginx"}, {"description": null, "name": "library/hello-world"}]}

And here is my nginx config:

upstream registry_http {
        server 192.168.1.61:5000;
}

server {
        listen 443 ssl;
        server_name registry.core.dckrindy.io;
        ssl_certificate_key /etc/nginx/ssl/letsencrypt/core.dckrindy.io/core.dckrindy.io.key;
        ssl_certificate /etc/nginx/ssl/letsencrypt/core.dckrindy.io/fullchain.cer;
        ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SH
A256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!
DES:!MD5:!PSK:!RC4";

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_session_cache shared:SSL:10m;

        chunked_transfer_encoding on;
        client_max_body_size 0;

        location / {
                auth_basic "registry.core.dckrindy.io";
                auth_basic_user_file /etc/nginx/htpasswd;

                proxy_pass http://registry_http;
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header Authorization "";
                proxy_read_timeout 900;
                client_max_body_size 0;
        }

        location /_ping {
                auth_basic off;
                proxy_pass http://registry_http;
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header Authorization "";
                proxy_read_timeout 900;
        }

        location /v1/_ping {
                auth_basic off;
                proxy_pass http://registry_http;
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header Authorization "";
                proxy_read_timeout 900;
        }
}

I run my v1 registry using:

docker run -d \
  --restart=always \
  -p 5000:5000 \
  --name registry \
  -v /data/registry:/tmp/registry \
  registry:0.9.1

[mgreenwald-pm]

@mbentley Thanks for all your help. I am going to manually pull and re-tag and push since we only have a few images anyways.