keel-hq / keel

Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates
https://keel.sh
Mozilla Public License 2.0
2.45k stars 282 forks source link

GCR service account unauthorised #623

Open StasCelium opened 3 years ago

StasCelium commented 3 years ago

Having a very similar problem to this issue https://github.com/keel-hq/keel/issues/546

After following the steps in the answer, i'm still having the same problem. Used keel previously many times and never had problems of this sort, not sure what is happening.

The error:

time="2021-06-19T06:25:43Z" level=error msg="trigger.poll.RepositoryWatcher.Watch: failed to add image watch job" error="Get \"https://gcr.io/v2/FOLDER/development/IMAGE_NAME/manifests/latest\": 

http: non-successful response (status=401 body=\"{\\\"errors\\\":[{\\\"code\\\":\\\"UNAUTHORIZED\\\"

,\\\"message\\\":\\\"You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication\\\"}]}\")

" image="namespace:dev, image:gcr.io/v2/FOLDER/development/IMAGE_NAME:latest,provider:kubernetes,trigger:poll,sched:@every 1m,secrets:[access-registry]"

I have installed via helm by typing helm upgrade --install keel --namespace=kube-system keel/keel --set googleApplicationCredentials=$(cat ./secret.json | base64) --set gcr.enabled=true --set gcr.projectId=PROJECT_ID

In my deployment file I have

      imagePullSecrets:
        - name: access-registry

I have also made sure that the service account does have the right premissions by doing

cat secret.json | docker login -u _json_key --password-stdin https://gcr.io

It did authenticate, and allowed me to push and pull images.

But still not working on k8s

EDIT: Whenever I kubectl apply -f mydeploy.yaml it seems to work fine, but whenever it needs to update an image, it just throws those errors

EDIT2:

level=warning msg="secrets.defaultGetter.lookupSecrets: secret found but couldn't detect authentication for the desired registry"
 image=gcr.io/PROJECT_ID/development/old_backend namespace=dev provider=kubernetes registry=gcr.io secrets="[access-registry]"
keyproco commented 3 years ago

Made it work after configured a notification channel weird..

StanislavBerezin commented 2 years ago

Can you elaborate how u fixed it?

Coz im facing exactly the same issue

StanislavBerezin commented 2 years ago

@rusenask pls help :)

keyproco commented 2 years ago

@StanislavBerezin I'm really sorry my friend, i didn't see the notification! I just got to setup Mattermost(It may be any notification setup), then everything goes well!

Try to look at your deployment manifest, you will find a section where you have to put credentials and the endpoint for the notification channel of any service, for my case, i used Mattermost, then this message just disappeared. I was like (what?) but didn't try to understand more as it works, but i believe this step was hanging the procedure