401 Unauthorized for correct password - and correct htpasswd.

[luke-park]

I'm receiving the following error message when attempting to login to a private registry from my local device:

login attempt to https://registry.redacted.com/v2/ failed with status: 401 Unauthorized

The logs from the registry server provide the following information:

redacted_1 | time="2018-07-15T00:22:09Z" level=warning msg="error authorizing context: basic authentication challenge for realm \"redacted\": invalid authorization credential" go.version=go1.7.6 http.request.host=redacted http.request.id=8d55e659-21df-426c-a0e8-35b9fcd4c345 http.request.method=GET http.request.remoteaddr="redacted:61605" http.request.uri="/v2/" http.request.useragent="docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))" instance.id=a1f3abc7-0531-45b1-913f-d2b7c712968b version=v2.6.2 redacted | redacted - - [15/Jul/2018:00:22:09 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))" redacted | time="2018-07-15T00:22:10Z" level=error msg="error authenticating user \"redacted\": authentication failure" go.version=go1.7.6 http.request.host=redactedhttp.request.id=60ba2ab8-068c-4951-b09d-313221899b28 http.request.method=GET http.request.remoteaddr="redacted:61606" http.request.uri="/v2/" http.request.useragent="docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))" instance.id=a1f3abc7-0531-45b1-913f-d2b7c712968b version=v2.6.2 redacted | time="2018-07-15T00:22:10Z" level=warning msg="error authorizing context: basic authentication challenge for realm \"redacted\": authentication failure" go.version=go1.7.6 http.request.host=redacted http.request.id=60ba2ab8-068c-4951-b09d-313221899b28 http.request.method=GET http.request.remoteaddr="redacted:61606" http.request.uri="/v2/" http.request.useragent="docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))" instance.id=a1f3abc7-0531-45b1-913f-d2b7c712968b version=v2.6.2 redacted | redacted - - [15/Jul/2018:00:22:10 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f40 kernel/4.9.87-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/18.03.1-ce \(windows\))"

I have confirmed that the registry is both aware of and definitely accessing my htpasswd file, as an invalid entry or filepath produces a different error log. Also of note is that the registry is running on a remote server with a valid hostname.

I have generated my htpasswd file using the command provided in the documentation: docker run --entrypoint htpasswd registry:2 -Bbn user password > htpasswd

I'm using the following Dockerfile to build the image, and the following docker-compose to run it:

# Pull the registry image.
FROM registry:2

# Add basic authentication.
RUN mkdir /auth
COPY ./htpasswd /auth/htpasswd

# Add TLS certs.
RUN mkdir /tls
COPY ./.crt /tls/.crt
COPY ./.key /tls/.key

# Expose 443.
EXPOSE 443

version: "3"
services:
    redacted:
        build: .
        environment:
            - REGISTRY_HTTP_ADDR=0.0.0.0:443
            - REGISTRY_HTTP_TLS_CERTIFICATE=/tls/.crt
            - REGISTRY_HTTP_TLS_KEY=/tls/.key
            - REGISTRY_AUTH=htpasswd
            - REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd
            - REGISTRY_AUTH_HTPASSWD_REALM=redacted
        ports:
            - "443:443"
        volumes:
            - /mnt/registry:/var/lib/registry

The TLS pair in use is a child certificate of a private CA - but this CA is trusted by the machine running the docker client attempting to login. I don't think it is a TLS-related issue as we seem to complete TLS handshake fine to receive a HTTP 401 error.

I am definitely using the correct password to login. What could be the problem here?