unable to upgrade code-server chart with keel

[grewhit25]

Details

microk8s v1.20.0 helm Version:"v3.0.2" GoVersion:"go1.13.5"

Thank you for this chart. I am unable to upgrade my k8s code-server instance using keel Seeing the following error in keel's log

Helm chart name and version:

nicholaswilde/code-server v0.1.0

Container name and tag:

linuxserver/code-server:v3.8.0-ls64

What steps did you take and what happened:

helm3 upgrade --install code-server nicholaswilde/code-server \ --namespace my-namespace \ --version 0.1.0 \ --set keel.policy="all",keel.trigger="poll",keel.pollSchedule="@every 10m" \ --set keel.images[0].repository="image.repository" \ --set keel.images[0].tag="image.tag" \ --reuse-values

keel’s log

time="2021-01-16T09:32:43Z" level=info msg="trigger.poll.RepositoryWatcher: new watch repository tags job added" digest="sha256:f048cd277c543d1694f8813a70b03b071335a4da1348a3bf5d5703f9537a7969" image="linuxserver/code-server:v3.8.0-ls64" job_name=ghcr.io/linuxserver/code-server schedule="@every 10m"

time="2021-01-16T09:33:04Z" level=info msg="preparing upgrade for code-server"
time="2021-01-16T09:33:04Z" level=info msg="reusing the old release's values"
time="2021-01-16T09:33:16Z" level=fatal msg="helm3: failed to update release from chart" error="template: code-server/templates/common.yaml:2:3: executing \"code-server/templates/common.yaml\" at <include \"common.all\" .>: error calling include: template: no template \"common.all\" associated with template \"gotpl\""

Causing keel to go into CrashLoopBackOff.

What did you expect to happen:

Successful upgrade to linuxserver/code-server:v3.8.0-ls65

Anything else you would like to add:

Not sure if the problem is with code-server chart or keel; but code-server is the only chart I have that is using the 'common' structure, so starting here.

Any pointer would be much appreciated.

Additional Information:

[nicholaswilde]

Was this an initial installation of the chart?

[grewhit25]

Thanks for your reply. No, this was not the initial installation of the code-server chart. I have however uninstalled the chart and re-installed with values for the previous container image then apply keel config so as to monitor the upgrade with the results as reported.

[nicholaswilde]

Let me look into this a little bit since I'm unfamiliar with Keel, which seems rather cool, by the way. Give me to the end of this weekend to see what I can come up with.

[grewhit25]

Update: Thank you so much for taking the timeout to investigate this issue especially on your weekend.

I tried upgrading one of my other chart; which is also based on the 'common' design. However, that also failed in a similar way.

So it looks very much like a possible issue with the way these particular type of charts behaves or possible deficiency with Keel.

time="2021-01-17T09:50:10Z" level=info msg="trigger.poll.RepositoryWatcher: new watch repository tags job added" digest="sha256:a5f50f10e183198a44a9d2d5aec5624ea55b849c7780d67e24e730b26874af50" image="homeassistant/home-assistant:2021.1.3" job_name=index.docker.io/homeassistant/home-assistant schedule="@every 10m"

time="2021-01-17T09:50:12Z" level=info msg="preparing upgrade for home-assistant"
time="2021-01-17T09:50:12Z" level=info msg="reusing the old release's values"
time="2021-01-17T09:50:13Z" level=fatal msg="helm3: failed to update release from chart" error="template: home-assistant/templates/common.yaml:2:4: executing \"home-assistant/templates/common.yaml\" at <include \"common.values.setup\" .>: error calling include: template: no template \"common.values.setup\" associated with template \"gotpl\""

Let me know what you think and how we should proceed.

btw, Keel is pretty cool; its been on my radar for sometime but only just got around to trialing it.

[nicholaswilde]

Here is what I've found. Keel's documentation uses the webhookdemo chart which specifies keel as a set of values in it's yaml file. The only documentation that I could find for the webhookdemo was here which the values.yaml file shows keel as a set of values.

# https://github.com/webhookrelay/webhook-demo/blob/master/webhookdemo/values.yaml
keel:
  # keel policy (all/major/minor/patch/force)
  policy: all
  # trigger type, defaults to events such as pubsub, webhooks
  trigger: poll
  # polling schedule
  pollSchedule: "@every 1m"
  # approvals required to proceed with an update
  approvals: 0
  # images to track and update

  images:
    - repository: image.repository
      tag: image.tag

Unfortunately, my charts, especially the common library chat, doesn't have the keel values. Looking at the keel documentation, you can add labels and annotations to the code-server Deployment and get keel to act on it.

# https://keel.sh/docs/#deployment-polling-example
...
  labels: 
      name: "wd"
      keel.sh/policy: force
      keel.sh/trigger: poll      
  annotations:
      keel.sh/pollSchedule: "@every 10m"
...

To get it to run on your already installed code-server chart, try creating this values.yaml file and running the below command.

# values.yaml
---
controllerAnnotations:
  keel.sh/pollSchedule: "@every 10m"

controllerLabels:
  keel.sh/policy: all
  keel.sh/trigger: poll

$ helm3 upgrade --install code-server nicholaswilde/code-server --namespace my-namespace --version 0.1.0 -f values.yaml --dry-run --debug

The above command should give you the following output.

...
---
# Source: code-server/templates/common.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: upgrade-code-server
  labels:
    helm.sh/chart: code-server-0.1.0
    app.kubernetes.io/name: code-server
    app.kubernetes.io/instance: upgrade
    app.kubernetes.io/version: "3.8.0"
    app.kubernetes.io/managed-by: Helm
    keel.sh/policy: all
    keel.sh/trigger: poll
  annotations:
    keel.sh/pollSchedule: '@every 10m'
...

To install it, simply remove the ---dry-run --debug from the above command. This assumes that you have the keel chart installed already.

$ helm3 upgrade --install code-server nicholaswilde/code-server --namespace my-namespace --version 0.1.0 -f values.yaml

All acceptable inputs for the values.yaml file can be found in the common library values.yaml file.

My setup: Helm: v3.5.0 Go: go1.15.6

[grewhit25]

Thank you for you detailed reply. I applied the annotations and labels as you suggested; great idea btw. However, and unfortunetly that did no work. Those updates, which I applied to an already installed chart as well as to a fresh install, just caused an error on keel with no upgrade.

Keel's log

time="2021-01-19T13:53:51Z" level=info msg="trigger.poll.RepositoryWatcher: new watch repository tags job added" digest="sha256:f048cd277c543d1694f8813a70b03b071335a4da1348a3bf5d5703f9537a7969" image="linuxserver/code-server:v3.8.0-ls64" job_name=ghcr.io/linuxserver/code-server schedule="@every 10m"
time="2021-01-19T13:54:01Z" level=warning msg="provider.kubernetes: got error while archiving approvals counter after successful update" error="approval not found: record not found" kind=deployment name=code-server namespace=my-namespace
time="2021-01-19T13:54:02Z" level=info msg="provider.kubernetes: resource updated" kind=deployment name=code-server namespace=my-namespace new=v3.8.0-ls65 previous=v3.8.0-ls64

I am thinking that the issue may be with Keel and the way it handles charts that are build using the 'common library' because all my other charts are upgading without any problems.

Anyway, thanks for all the help you have given me on this one and I am happy for you to close the issue.

[nicholaswilde]

It looks like it successfully updated the chart from v3.8.0-ls64 to v3.8.0-ls65 per the last line of your logs. I found this issue at keel that says the error is just a warning. Closing out the issue.

[grewhit25]

Oops! My bad; I have gone back and double checked and indeed the chart did get updated. So strike my last comment, I simply took the error as final without fully considering the outcome.

Thanks again.

[nicholaswilde]

I'm glad that it's working for you. Happy helming.