keel-hq / keel

Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates
https://keel.sh
Mozilla Public License 2.0
2.45k stars 282 forks source link

GCR as Private Registry when Not On GKE #463

Open grifonas opened 4 years ago

grifonas commented 4 years ago

Hi, I've been having trouble configuring image polling with GCR. We are using GCR as our Docker registry however the cluster doesn't live on Google Cloud.

Pub Sub I tried pub/sub, but that appears to require instance metadata, which, obviously, my cluster nodes don't have.

Polling

Configuration:

apiVersion: v1
imagePullSecrets:
- name: gcr-json-key
kind: ServiceAccount
metadata:
  labels:
    app: keel
    chart: keel-0.8.16
    heritage: Tiller
    release: keel
  name: keel

Keel logs:

time="2019-12-09T13:14:37Z" level=warning msg="secrets.defaultGetter: registry not found among secrets" image="keelhq/keel:0.15.0-rc1" secret_ref=gcr-json-key
time="2019-12-09T13:14:37Z" level=warning msg="secrets.defaultGetter.lookupSecrets: secret found but couldn't detect authentication for the desired registry" image=index.docker.io/keelhq/keel namespace=kube-system provider=helm registry=index.docker.io secrets="[gcr-json-key]"
time="2019-12-09T13:14:38Z" level=info msg="trigger.poll.RepositoryWatcher: new watch repository tags job added" digest="sha256:28a9226ca76b99d845fab6ea43b46a1cf1695ccbebd6d1693c2a871c4624334d" image="keelhq/keel:0.15.0-rc1" job_name=index.docker.io/keelhq/keel schedule="@every 3m"
time="2019-12-09T13:14:38Z" level=warning msg="secrets.defaultGetter: registry not found among secrets" image="keelhq/keel:0.15.0-rc1" secret_ref=gcr-json-key
time="2019-12-09T13:14:38Z" level=warning msg="secrets.defaultGetter.lookupSecrets: secret found but couldn't detect authentication for the desired registry" image=index.docker.io/keelhq/keel namespace=kube-system provider=helm registry=index.docker.io secrets="[gcr-json-key]"

☝️ This seems to indicate that Keel is able to detect and use the secret.

Is there a way to use polling with GCR?

Thank you!

rchenzheng commented 4 years ago

Did you ensure the service account has storage viewer role to the registry?

kenneyfortressiq commented 4 years ago

@rchenzheng I have pub/sub editor role and it has a parsing bug with index.docker.io. All my images are in GCR including keel. It is annoying and leads to me thinking it is failing to update..

steps to reproduce: 1) 0.16.0 chart 2) have value.yaml below that is working on GKE helm chart 3) change cluster name and move to Azure AKS 4) run: helm upgrade --install keel . --namespace kube-system -f values.yaml 5) see issue

error: time="2020-04-14T23:15:07Z" level=debug msg="performing scan" time="2020-04-14T23:15:07Z" level=debug msg="registry index.docker.io is not a GCR, skipping"

values.yaml

polling:
  enabled: false

gcr:
  enabled: true
  projectId: "xxx"
  clusterName: "tr-us-000"
keel:
  # keel policy (all/major/minor/patch/force)
  policy: all
  # trigger type, defaults to events such as pubsub, webhooks
  trigger: poll
  # polling schedule
  pollSchedule: "@every 3m"
  # images to track and update
  images:
    - repository: gcr.io/..
      tag: latest