Helm Installation with Cosign Validator Not Working

[kristi-balla]

Describe the bug I want to install connaisseur in my AWS backed minikube k8s cluster. I want to use cosign as the validator. When installing via helm with the updated values.yaml, I get the following error:

Error: INSTALLATION FAILED: template: connaisseur/templates/secrets.yaml:3:41: executing "connaisseur/templates/secrets.yaml" at <.auth>: wrong type for value; expected map[string]interface {}; got interface {}

Expected behavior The installation should complete without errors

Optional: To reproduce

1. change values.yaml like so:

key="$COSIGN_PUBLIC_KEY" yq --inplace '.application.validators += [{"name":"cosignvalidator", "type":"cosign", "trustRoots":[{"name":"default", "key" : strenv(key) }]}]' values.yaml

The validator should look like this when done:

name: cosignvalidator
type: cosign
trustRoots:
  - name: default
    key: |-
      -----BEGIN PUBLIC KEY-----
      MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE/KtRQ13FLpPx1kk1tN1CvuFshEzb
      8CbEcnnfiZ7fH/YdoAjc2vqPb5G/tVyELTCKh872hHwXke4BTaEUKfsFCg==
      -----END PUBLIC KEY-----

--> The "-" at key has nothing to do with the bug. Removing it doesn't solve my problem.

2. add a policy:

yq --inplace '.application.policy += {"pattern": "local.registry:5000/*", "validator": "cosignvalidator"}' values.yaml

The local.registry is a locally running docker registry from docker run -d -p 5000:5000 --name registry registry:2. I edited the hosts file accordingly and the registry works as expected. No credentials are configured for the registry.

3. install via:

helm install connaisseur helm --atomic --namespace connaisseur

Optional: Versions (please complete the following information as relevant):

• OS: debian 12
• Kubernetes Cluster: v1.31.0, started via minikube with:
  • cni=calico
  • driver=docker
  • container-runtime=containerd
  • insecure-registries=local.registry:5000
• Connaisseur: 3.7.0

[kristi-balla]

This is how the templates/secrets.yaml looks like:

{{ $root := . -}}
{{ range .Values.application.validators -}}
{{ if and (hasKey . "auth") (and (hasKey .auth "username") (hasKey .auth "password") (or (eq .type "notaryv1") (hasKey .auth "registry"))) -}}
apiVersion: v1
kind: Secret
metadata:
  name: {{ .name }}-auth
  namespace: {{ $root.Release.Namespace }}
  labels:
    {{- include "connaisseur.labels" $root | nindent 4 }}
type: Opaque
data:
  secret.yaml: {{ .auth | toYaml | b64enc }}
---
{{- end }}
{{- end }}

My hunch is telling me that the if condition does not evaluate to true for the custom cosign validator. It does not have an auth section and it is not of notaryv1 type. Changing this wouldn't be difficult and I'd be happy to submit a PR, but I wanted to be sure this is the cause of the error.

[phbelitz]

@kristi-balla I tried to replicate this and did not run into any errors ... here my sample validators + policies:

  validators:
    - name: allow
      type: static
      approve: true
    - name: deny
      type: static
      approve: false
    - name: dockerhub
      type: notaryv1
      trustRoots:
        - name: default # root from dockerhub
          key: |
            -----BEGIN PUBLIC KEY-----
            MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEOXYta5TgdCwXTCnLU09W5T4M4r9f
            QQrqJuADP6U7g5r9ICgPSmZuRHP/1AYUfOQW3baveKsT969EfELKj1lfCA==
            -----END PUBLIC KEY-----
        - name: sse # root from sse
          key: |
            -----BEGIN PUBLIC KEY-----
            MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEsx28WV7BsQfnHF1kZmpdCTTLJaWe
            d0CA+JOi8H4REuBaWSZ5zPDe468WuOJ6f71E7WFg3CVEVYHuoZt2UYbN/Q==
            -----END PUBLIC KEY-----
    - name: cosignvalidator
      type: cosign
      trustRoots:
        - name: default
          key: |-
            -----BEGIN PUBLIC KEY-----
            MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEvtc/qpHtx7iUUj+rRHR99a8mnGni
            qiGkmUb9YpWWTS4YwlvwdmMDiGzcsHiDOYz6f88u2hCRF5GUCvyiZAKrsA==
            -----END PUBLIC KEY-----
  # policy options: https://sse-secure-systems.github.io/connaisseur/latest/basics/#configuration-options_2
  # explanation of policy patterns: https://sse-secure-systems.github.io/connaisseur/latest/basics/#image-policy
  policy:
    - pattern: "*:*"
      validator: deny
    - pattern: "docker.io/library/*:*"
      validator: dockerhub
    - pattern: "docker.io/securesystemsengineering/*:*"
      validator: dockerhub
      with:
        trustRoot: sse
    - pattern: "registry.k8s.io/*:*"
      validator: allow
    - pattern: "docker.io/securesystemsengineering/testimage:co-*"
      validator: cosignvalidator
      with:
        verifyInTransparencyLog: false

The secrets.yaml should not get created if you didn't define an auth for the validator ... maybe there are still some parts remaining from a previous installation ? Try running make annihilate (given you have cmake installed) to erase all traces of Connaisseur and then do a clean install.

[kristi-balla]

@phbelitz thank you for your prompt response! The secrets.yaml is by default in the repository. Deleting it just invites other errors about missing auth fields. Deleting the templates folder does fix the installation, but no pods are started at all.

I even tried a fresh installation and it keeps failing with that error message. I am probably missing something trivial, but I do not know what

[kristi-balla]

After some digging around, I managed to somehow circumvent this. I was running on helm v3.7.0 and updating to the latest v3.16.3 magically made my problem go away. I wouldn't call this a solution, but at least we can close the issue