[Feature Request] Implementing a path keeping "X-registry-Auth" header untouched

[hilschernetpi]

We are successfully using proxy function to local /var/run/docker.sock. Exception: some functions (like a push image) needing a "X-registry-Auth" header in the request fail with "authentication denied" in our use case. Here is the reason: The header is manipulated by createRegistryAuthenticationHeader() in file source code /proxy/registry.go. We see manipulations in two cases if given serverAddress == "" and if given serveraddress equals to any address found in the registry list. But in our case the registry list is empty (not set any) and also our given serveraddress in the request is != "" and we have not defined DockerHub credentials at all. Since all if clauses fail the function createRegistryAuthenticationHeader returns an authenticationHeader at value nil.

A better approach would be to keep the header untouched if neither the serverAddress == "" and no registry set was found matching. This allows to transparently use the proxy function on "X-registry-Auth" needing function without having a need to define any credentials in portainer before. Proposal: Instead of handing over just the originalHeaderData.Serveraddress to the function createRegistryAuthenticationHeader , it would be better to hand over the complete originalHeaderData object and return it back in case the two if case checks fail.

[deviantony]

Hi @hilschernetpi

I'm pretty curious about your use case, are you leveraging Docker API through Portainer API to orchestrate your Docker management?

[hilschernetpi]

Dear Anthony,

we are manufacturing the Industrial Raspberry Pi 3 named netPI https://www.netiot.com/netpi/industrial-raspberry-pi-3/.

Since the beginning this device includes a secured Linux and three additional softwares only: Docker, Portainer.io and a management web GUI. The dockerd version is little special. It cannot map the /dev/sda1/, has restrictions concerning privileged mode and others. Also this device has no Linux ssh console access. Every software extension is done through containers only giving the end customers the chance to implement their own software on top of this “secure” device. Nobody is able to compromise the Linux with this architecture. This is how we see “open innovation”. Hilscher provides a hardware and customer completes it to an end device.

Now the point is that we used portainer 1.12.4 since the beginning till end of last year. We provided HTTP API examples for portainer.io (1.12.4, even not documented officially) in order to let customer load fresh netPI devices with their own software during their manufacturing process. So they are getting devices from us which are of course “empty” and not loaded with any containers. Then they start a http client using netPI’s portainer.io HTTP API to preload the devices with their containers. So no human has a need to use netPI’s Web GUI manually, everything goes automatically.

Since December we are using portainer 1.19.x. Of course the API has changed and we updated our examples. The examples includes portainer authentication, pulling images, tagging images, etc. and also pushing images. Pushing images to push modified and tagged containers to a registry. And now comes the point: while on 1.12.4 pushing worked fine we detected that pushing under currently used version 1.19.x does no longer works. If we access dockerd socket directly with “curl” command and x-auth it worked, but not through portainer.io API. And then we were looking for the reason and found the x-registry-auth manipulation in portainer source.

I understand why you have implemented it. You want portainer.io work as a proxy using the credentials configured. In our use case there is no “human” and nobody will enter any credentials, we want indeed use dockerd and its API 100% transparently or in your words “orchestration” from remote. Over a VPN connection for example if the devices are finally installed in a control cabinet and doing their job.

Thx Armin

[deviantony]

Hi Armin, thanks for the detailed response.

Could you share with me an example of an API request that is failing for you?

[hilschernetpi]

Hello Antony,

do you know Node-RED tool?

Since we deliver the docker/portainer.io API example to customers as a Node-RED flow, I would a hand over to you an adapted flow that runs against your demo.portainer.io web site.

Next to pull, tag etc. accesses you find a push to a dockerhub registry in there as well. This fails cause of the x-registry-auth problem. A workaround is also implemented: We implemented a "create registry" entry as well. Then portainer.io knows the registry, fills out the x-registry-auth in the push correctly, and then the image push command works ... but the goal is to use it without "creating a registry"

Armin

[deviantony]

Hi @hilschernetpi

I am not aware of the Node-RED tool. I can see what your problem is I think I just wanted to check if you had access to the specific HTTP request that is failing to help me reproduce it.

[hilschernetpi]

Dear Antony,

here a sequence of bash command you can call in a linux console to get to the problem

// SET variables for portainer !!! ENDPOINT is set fix to 0 cause it fits do demo.portainer.io!!!
USERNAME=admin
PASSWORD=tryportainer
URL='http://demo.portainer.io'
// Login to demo (returns JWT token)
JWT=$(curl --insecure -d '{"Username": "'${USERNAME}'", "Password": "'${PASSWORD}'"}' -X POST ${URL}'/api/auth' | cut -d"\"" -f4)
// Pull a simple image e.g. resin/raspberrypi3-alpine
curl -H 'Authorization:Bearer '${JWT}'' -X POST ${URL}'/api/endpoints/0/docker/images/create?fromImage=resin/raspberrypi3-alpine&tag=latest'
// SET variables for docker registry e.g. dockerhub
USERNAME_REGISTRY=superjojo2001
PASSWORD_REGISTRY= <send to you by email separately>
REGESTRY_URL="registry.hub.docker.com"
REPO="superjojo2001/mytest"
TAG="latest"
AUTH=$(echo '{"username": "'${USERNAME_REGISTRY}'", "password": "'${PASSWORD_REGISTRY}'", "serveraddress": "'${REGESTRY_URL}'"}' | base64 | sed ':a;N;$!ba;s/\n//g' )
// Create Container 'myContainer' from 'resin/raspberrypi3-alpine'
CONTAINER_ID=$(curl -H 'Authorization: Bearer '${JWT}'' -H "Content-Type: application/json" -d '{"Image":"resin/raspberrypi3-alpine"}' -X POST ${URL}'/api/endpoints/0/docker/containers/create?name=myContainer' | cut -d"\"" -f4)
// Commit Container to have a tag that fits to the dockerhub repo we want  to push
QUERY='?container='$CONTAINER_ID'&repo='${REPO}'&tag='${TAG}''
curl -H 'Authorization: Bearer '${JWT}'' -X POST ${URL}'/api/endpoints/0/docker/commit'${QUERY}''
// Push image to e.g. dockerhub and the error appears
curl -H 'Authorization: Bearer '${JWT}'' -H 'X-Registry-Auth: '${AUTH}'' -X POST ${URL}/api/endpoints/0/docker/images/${REPO}:${TAG}/push

Call one command after the other. Change the URL to your local settings if you don't want to test it against demo.portainer.io. The required password for pushing it to my private repositry I will be sending to you in a separate email.

Thx Armin