Kong chart cannot be deployed with ArgoCD / Kustomize

[jamesdkelly88]

Currently the kong chart template helper uses capabilities to determine if gateway api resources are installed:

{{- if or (.Capabilities.APIVersions.Has "gateway.networking.k8s.io/v1alpha2") (.Capabilities.APIVersions.Has "gateway.networking.k8s.io/v1beta1") (.Capabilities.APIVersions.Has "gateway.networking.k8s.io/v1")}}

Because of this, any deployments that use helm templating (I've tried kustomize --enable helm and ArgoCD) don't include the resources in the cluster role definition, so the routes don't work and the proxy logs are full of this error:

E0218 17:09:52.151920 1 reflector.go:147] pkg/mod/k8s.io/client-go@v0.28.3/tools/cache/reflector.go:229: Failed to watch *v1.Gateway: failed to list *v1.Gateway: gateways.gateway.networking.k8s.io is forbidden: User "system:serviceaccount:kong:kong-kong" cannot list resource "gateways" in API group "gateway.networking.k8s.io" at the cluster scope

It would be helpful to have a way of overriding this for these situations

[pmalek]

I recall having the same problem and what you should look at is helm template's --api-versions arg.

  -a, --api-versions strings                       Kubernetes api versions used for Capabilities.APIVersions

this can be used like so:

... --api-versions="networking.k8s.io/v1/Ingress" ... 

This allows you to inject any api version which will be used by helm when not run against a cluster (e.g. that's what helm template is for)

There has to be an option to specify that in Argo CD.

[jamesdkelly88]

I can't find any documentation on how to pass this to helm from an argocd application definition. Do you have an example?

[pmalek]

It seems that this is still an unsolved problem: https://github.com/argoproj/argo-cd/issues/7291

Docs - https://argo-cd.readthedocs.io/en/stable/user-guide/helm/#build-environment - also do not mention this anyhow.

I believe we can consider adding the these resource policy rules unconditionally. They'd just end up as noop in cluster that do not define those resources and will be rendered properly when used with tools like Argo.

I'm not sure yet if that's the correct approach so not proposing a PR just yet. LMKWYT

[jamesdkelly88]

That's one way to go. I was thinking that the if statement could have an extra option in there from the values file, which is defaulted to false so it doesn't break anything.

[jamesdkelly88]

So it looks like you can pass the apiVersions option in if using the helmCharts section of a kustomization, which will work well enough for me.

[pmalek]

I'm glad you've figured it out.

Can you provide a redacted snippet for future readers?

[jamesdkelly88]

helmCharts:
- name: kong
  repo: https://charts.konghq.com
  releaseName: kong
  namespace: kong
  version: 2.35.1
  includeCRDs: true
  valuesFile: values.yaml
  apiVersions:
    - gateway.networking.k8s.io/v1