keel-hq / keel

Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates
https://keel.sh
Mozilla Public License 2.0
2.45k stars 282 forks source link

Azure Polling -> failed to get image digest #711

Open lmarkusl opened 1 year ago

lmarkusl commented 1 year ago

Hi all,

we are moved into the container registry on azure and we get keel running with webhooks for azure.

Now we want to use Polling.

The images could be pulled with the same docker config secret. But if keel wants to access the manifest it saying its not allowed:

Could it be that is related to how Azure wants the access_token?

https://github.com/Azure/acr/blob/main/docs/Token-BasicAuth.md ?

thx markus

` time="2023-03-22T07:21:42Z" level=debug msg="extension.credentialshelper: helper doesn't support this registry" error="unsupported registry" helper=aws tracked_image="namespace:default,image:crexample.azurecr.io/test/nginx:1.14.2,provider:kubernetes,trigger:poll,sched:@every 1m,secrets:[keeltest-cr-secret]"

time="2023-03-22T07:21:42Z" level=debug msg="secrets.defaultGetter: secret looked up successfully" image=crexample.azurecr.io/test/nginx namespace=default provider=kubernetes registry=crexample.azurecr.io

time="2023-03-22T07:21:42Z" level=debug msg="registry.manifest.head url=https://crexample.azurecr.io/v2/test/nginx/manifests/1.14.2 repository=test/nginx reference=1.14.2"

time="2023-03-22T07:21:42Z" level=error msg="trigger.poll.RepositoryWatcher.addJob: failed to get image digest" error="Get \"https://crexample.azurecr.io/v2/test/nginx/manifests/1.14.2\": http: non-successful response (status=401 body=\"{\\"errors\\":[{\\"code\\":\\"UNAUTHORIZED\\",\\"message\\":\\"authentication required, visit https://aka.ms/acr/authorization for more information.\\",\\"detail\\":[{\\"Type\\":\\"repository\\",\\"Name\\":\\"test/nginx\\",\\"Action\\":\\"pull\\"}]}]}\n\")" image="test/nginx:1.14.2" password="****" username=testkeel

time="2023-03-22T07:21:42Z" level=error msg="trigger.poll.RepositoryWatcher.Watch: failed to add image watch job" error="Get \"https://crexample.azurecr.io/v2/test/nginx/manifests/1.14.2\": http: non-successful response (status=401 body=\"{\\"errors\\":[{\\"code\\":\\"UNAUTHORIZED\\",\\"message\\":\\"authentication required, visit https://aka.ms/acr/authorization for more information.\\",\\"detail\\":[{\\"Type\\":\\"repository\\",\\"Name\\":\\"test/nginx\\",\\"Action\\":\\"pull\\"}]}]}\n\")" image="namespace:default,image:crexample.azurecr.io/test/nginx:1.14.2,provider:kubernetes,trigger:poll,sched:@every 1m,secrets:[keeltest-cr-secret]" `

el-aasi commented 1 year ago

How do you configure the docker secrets for ACR?

We had a similar issue, where we configured the docker secrets as described in the MS documentation using the service principle and client id, and it was failing very similar to what you are presenting.

To solve we have recreated the docker secret using the credentials from ACR -> Access Keys (Username and password) and it got it working. The thing is that we are using it in an AKS context, so we didn't have an image pull secret before at all as you can setup aks->acr implicit authorization using the managed identities, but that of course didn't work for Keel as it needs explicit secrets specified.

I feel that maybe this should be documented somewhere?

philipp-durrer-jarowa commented 3 weeks ago

Anybody up for adding managed identity support?