encountered errors while adding images: status=401 authentication required

[shoce]

We are using a self-hosted Gitlab instance and store the docker images in the registry provided by the Gitlab. Using microk8s on an Ubuntu machine. We have all the secrets connected to the deployment and k8s downloads the image successfully. But keel has troubles with accessing the registry. Any leads how to fix?

# kubectl describe -n dev deployment/dev-mobile

Name:                   dev-mobile
Namespace:              dev
CreationTimestamp:      Fri, 20 Aug 2021 18:12:47 +0200
Labels:                 app=dev-mobile
Annotations:            deployment.kubernetes.io/revision: 1
                        keel.sh/policy: glob:*.*
                        keel.sh/pollSchedule: @every 10s
                        keel.sh/trigger: poll
Selector:               app=dev-mobile
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=dev-mobile
  Containers:
   dev-mobile:
    Image:      registry.mydomain.com/front/mobile/develop:0210819.813
    Port:       80/TCP
    Host Port:  0/TCP
    Liveness:   exec [/bin/sh -c test z$$(curl -A healthcheck -s -L http://127.0.0.1/mobile.commit.txt) = z$$(cat /var/www/mobile/mobile.commit.txt)] delay=5s timeout=1s period=3s #success=1 #failure=3
    Environment Variables from:
      dev-mobile-config  ConfigMap  Optional: false
    Environment:            <none>
    Mounts:                 <none>
  Volumes:                  <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  <none>
NewReplicaSet:   dev-mobile-8955bcf8b (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  49s   deployment-controller  Scaled up replica set dev-mobile-8955bcf8b to 1

# kubectl logs -n kube-system deployment/keel:

time="2021-08-20T16:10:18Z" level=error msg="trigger.poll.manager: got error(-s) while watching images" error="encountered errors while adding images: Get \"https://registry.mydomain.com/v2/front/mobile/develop/manifests/0210819.813\": http: non-successful response (status=401 body=\"{\\\"errors\\\":[{\\\"code\\\":\\\"UNAUTHORIZED\\\",\\\"message\\\":\\\"authentication required\\\",\\\"detail\\\":[{\\\"Type\\\":\\\"repository\\\",\\\"Class\\\":\\\"\\\",\\\"Name\\\":\\\"front/mobile/develop\\\",\\\"Action\\\":\\\"pull\\\"}]}]}\\n\")"

[kosgug]

I got the error code : 403

http: non-successful response (status=403 body=\"{\\\"errors\\\":[{\\\"code\\\":\\\"DENIED\\\",\\\"message\\\":\\\"invalid token\\\"}]}\\n\")"

[shoce]

@kosgug This issue looks like a different type: 403 usually means that authentication was successful but authorization failed. In my case there is 401 status that means the credentials were not sent by the registry client (keel) or not accepted by the registry server.