traefik / traefik-helm-chart

Traefik Proxy Helm Chart
https://traefik.io
Apache License 2.0
1.08k stars 762 forks source link

TCPRoute CRD missing on clean install #1253

Closed strophy closed 1 hour ago

strophy commented 1 hour ago

Welcome!

What version of the Traefik's Helm Chart are you using?

33.0.0

What version of Traefik are you using?

3.2.0

What did you do?

I'm installing Traefik with Argo CD with following config:

# Chart.yaml
apiVersion: v2
name: traefik
type: application
version: 1.0.0
appVersion: "3.2.0"
dependencies:
  - name: traefik
    version: 33.0.0
    repository: https://traefik.github.io/charts
# values.yaml
traefik:
  providers:
    kubernetesIngress:
      enabled: false
    kubernetesGateway:
      enabled: true
      experimentalChannel: true
  gateway:
    enabled: true
    annotations:
      cert-manager.io/issuer: selfsigned-issuer
    listeners:
      websecure:
        port: 8443
        hostname: "*.bioapp.internal"
        protocol: HTTPS
        namespacePolicy: All
        certificateRefs:
          - name: bioapp-tls
      mqtt:
        port: 1883
        protocol: TCP
        namespacePolicy: All
  ports:
    mqtt:
      port: 1883
      protocol: TCP

And trying to create a TCPRoute as follows:

# tcproute.yaml
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TCPRoute
metadata:
  name: mqtt
spec:
  parentRefs:
    - name: traefik-gateway
      namespace: traefik
  hostnames:
    - {{ .Values.tcproute.mqtt }}
  rules:
    - backendRefs:
        - name: emqx
          namespace: bioapp
          port: 1883

What did you see instead?

Argo CD fails to create the TCPRoute resource with the error The Kubernetes API could not find gateway.networking.k8s.io/TCPRoute for requested resource bioapp/mqtt. Make sure the "TCPRoute" CRD is installed on the destination cluster.

Traefik logs the following errors:

W1115 15:44:20.132037       1 reflector.go:561] k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: failed to list *v1alpha3.BackendTLSPolicy: the server could not find the requested resource (get backendtlspolicies.gateway.networking.k8s.io)
E1115 15:44:20.132251       1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: Failed to watch *v1alpha3.BackendTLSPolicy: failed to list *v1alpha3.BackendTLSPolicy: the server could not find the requested resource (get backendtlspolicies.gateway.networking.k8s.io)" logger="UnhandledError"
W1115 15:44:28.457977       1 reflector.go:561] k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: failed to list *v1alpha2.TLSRoute: the server could not find the requested resource (get tlsroutes.gateway.networking.k8s.io)
E1115 15:44:28.458046       1 reflector.go:158] "Unhandled Error" err="k8s.io/client-go@v0.31.1/tools/cache/reflector.go:243: Failed to watch *v1alpha2.TLSRoute: failed to list *v1alpha2.TLSRoute: the server could not find the requested resource (get tlsroutes.gateway.networking.k8s.io)" logger="UnhandledError"

The following CRDs seem to have been installed:

$ kubectl get crds | grep 'traefik\|networking\|gateway'
accesscontrolpolicies.hub.traefik.io                    2024-10-29T10:17:40Z
apiaccesses.hub.traefik.io                              2024-10-29T10:17:41Z
apibundles.hub.traefik.io                               2024-10-29T10:17:41Z
apiplans.hub.traefik.io                                 2024-10-29T10:17:41Z
apiportals.hub.traefik.io                               2024-10-29T10:17:40Z
apiratelimits.hub.traefik.io                            2024-10-29T10:17:41Z
apis.hub.traefik.io                                     2024-10-29T10:17:40Z
apiversions.hub.traefik.io                              2024-10-29T10:17:40Z
gatewayclasses.gateway.networking.k8s.io                2024-10-29T10:17:41Z
gateways.gateway.networking.k8s.io                      2024-10-29T10:17:41Z
grpcroutes.gateway.networking.k8s.io                    2024-10-29T10:17:41Z
httproutes.gateway.networking.k8s.io                    2024-10-29T10:17:41Z
ingressroutes.traefik.io                                2024-10-29T10:17:40Z
ingressroutetcps.traefik.io                             2024-10-29T10:17:41Z
ingressrouteudps.traefik.io                             2024-10-29T10:17:40Z
middlewares.traefik.io                                  2024-10-29T10:17:41Z
middlewaretcps.traefik.io                               2024-10-29T10:17:40Z
referencegrants.gateway.networking.k8s.io               2024-10-29T10:17:40Z
serverstransports.traefik.io                            2024-10-29T10:17:40Z
serverstransporttcps.traefik.io                         2024-10-29T10:17:40Z
tlsoptions.traefik.io                                   2024-10-29T10:17:40Z
tlsstores.traefik.io                                    2024-10-29T10:17:41Z
traefikservices.traefik.io                              2024-10-29T10:17:41Z

What is your environment & configuration?

k3s v1.30.6+k3s1 Argo CD v2.12.6

Additional Information

I have read several similar issues, but they seem mostly related to difficulty when upgrading, but this is a clean deployment on a new cluster. It seems strange to me that the grpcroutes.gateway.networking.k8s.io CRD is created but tcproutes is not, since they are both experimental channel. The error logged by the Traefik container seems related to missing TLSRoutes, and again similar issues seemed to mention some missing RBAC, but if I am reading the other issues correctly this should have been resolved in the 33.0.0 release. Thanks for any help, I'm new to this so expect I've made a mistake somewhere.

strophy commented 1 hour ago

I did indeed make a mistake, the solution is in the documentation. It is necessary to separately installed CRDs for experimental channel resources as follows:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.0/experimental-install.yaml