search

l7mp / stunner

A Kubernetes media gateway for WebRTC. Contact: info@l7mp.io
https://l7mp.io
MIT License
751 stars 58 forks source link

Unable to start pod due to wrong CRD version #171

Closed hudson-s closed 4 weeks ago

hudson-s commented 1 month ago

Description

[Description of the problem]

Steps to Reproduce

[Brief description of the steps you took to encounter the problem, if applicable]

Expected behavior: [What you expected to happen]

Actual behavior: [What actually happened]

Versions

[Which version of STUNner you are using]

Info

[Please copy-paste the output of the below commands and make sure to remove all sensitive information, like usernames, passwords, IP addresses, etc.]

Gateway API status

[Output of kubectl get gateways,gatewayconfigs,gatewayclasses,udproutes.stunner.l7mp.io --all-namespaces -o yaml]

Operator logs

[Output of kubectl -n stunner-system logs $(kubectl get pods -l control-plane=stunner-gateway-operator-controller-manager --all-namespaces -o jsonpath='{.items[0].metadata.name}')] 024-10-22T11:25:56.835450039Z INFO setup starting operator thread 2024-10-22T11:25:56.835501009Z INFO gatewayconfig-controller Created GatewayConfig controller 2024-10-22T11:25:56.835512302Z INFO gatewayconfig-controller Watching GatewayConfig objects 2024-10-22T11:25:56.846979851Z INFO gatewayconfig-controller Watching Secret objects 2024-10-22T11:25:56.847008073Z INFO dataplane-controller created dataplane controller 2024-10-22T11:25:56.847013364Z INFO dataplane-controller watching dataplane objects 2024-10-22T11:25:56.847019738Z INFO gateway-controller Created Gateway controller 2024-10-22T11:25:56.847025445Z INFO gateway-controller Watching GatewayClass objects 2024-10-22T11:25:56.847029244Z INFO gateway-controller Watching Gateway objects 2024-10-22T11:25:56.856714629Z ERROR setup problem running operator {"error": "Cannot register gateway controller: no matches for kind \"Gateway\" in version \"gateway.networking.k8s.io/v1\""}

levaitamas commented 1 month ago

hi @hudson-s !

This error is the key:

{"error": "Cannot register gateway controller: no matches for kind "Gateway" in version "gateway.networking.k8s.io/v1""}

I guess Gateway API v1 CRD is somehow not installed in your cluster (install help: https://gateway-api.sigs.k8s.io/guides/#install-standard-channel).

Is this a new STUNner install or you just updated from an earlier version? BTW mind to tell a few words about your use-case?

hudson-s commented 1 month ago

hi @hudson-s !

This error is the key:

{"error": "Cannot register gateway controller: no matches for kind "Gateway" in version "gateway.networking.k8s.io/v1""}

I guess Gateway API v1 CRD is somehow not installed in your cluster (install help: https://gateway-api.sigs.k8s.io/guides/#install-standard-channel).

Is this a new STUNner install or you just updated from an earlier version? BTW mind to tell a few words about your use-case?

I am using the helm source file for a new installation

  1. cd /stunner-helm-main/helm/stunner-gateway-operator
  2. helm install stunner-gateway-operator . --create-namespace --namespace=stunner-system In addition:

kubectl api-resources | grep gateway gatewayclasses gc gateway.networking.k8s.io/v1beta1 false GatewayClass gateways gtw gateway.networking.k8s.io/v1beta1 true Gateway grpcroutes gateway.networking.k8s.io/v1alpha2 true GRPCRoute httproutes gateway.networking.k8s.io/v1beta1 true HTTPRoute referencegrants refgrant gateway.networking.k8s.io/v1beta1 true ReferenceGrant tcproutes gateway.networking.k8s.io/v1alpha2 true TCPRoute tlsroutes gateway.networking.k8s.io/v1alpha2 true TLSRoute

levaitamas commented 1 month ago

Which STUNner version is it? We upgraded the Gateway API CRD in our helm chart to v1.0.0 10 month ago: https://github.com/l7mp/stunner-helm/blob/main/helm/stunner-gateway-operator/crds/gateway-api-crd.yaml

Can you give info on your cluster? Is it self-managed or do you use managed K8s (e.g., GKE, AKS, EKS, etc.)?

hudson-s commented 1 month ago

Which STUNner version is it? We upgraded the Gateway API CRD in our helm chart to v1.0.0 10 month ago: https://github.com/l7mp/stunner-helm/blob/main/helm/stunner-gateway-operator/crds/gateway-api-crd.yaml

Can you give info on your cluster? Is it self-managed or do you use managed K8s (e.g., GKE, AKS, EKS, etc.)? I am use managed by alibaba cloud k8s. I am using the latest clone

image

levaitamas commented 1 month ago

Interesting, can you double check that the CRD install was successful?

To install the CRDS, Just run kubectl apply -f helm/stunner-gateway-operator/crds/ and check the output, and the available API resources and CRDs in your cluster. I have a new insall on AKS, and the output is the following:

kubectl api-resources | grep gateway
gatewayclasses                      gc                  gateway.networking.k8s.io/v1         false        GatewayClass
gateways                            gtw                 gateway.networking.k8s.io/v1         true         Gateway
grpcroutes                                              gateway.networking.k8s.io/v1alpha2   true         GRPCRoute
httproutes                                              gateway.networking.k8s.io/v1         true         HTTPRoute
referencegrants                     refgrant            gateway.networking.k8s.io/v1beta1    true         ReferenceGrant
tcproutes                                               gateway.networking.k8s.io/v1alpha2   true         TCPRoute
tlsroutes                                               gateway.networking.k8s.io/v1alpha2   true         TLSRoute
udproutes                                               gateway.networking.k8s.io/v1alpha2   true         UDPRoute
gatewayconfigs                      gwconf              stunner.l7mp.io/v1                   true         GatewayConfig

Note that we also see gatewayconfigs.stunner.l7mp.io/v1 resources in the output.

kubectl get crds | grep gateway
gatewayclasses.gateway.networking.k8s.io         2024-10-21T17:55:51Z
gatewayconfigs.stunner.l7mp.io                   2024-10-21T17:55:51Z
gateways.gateway.networking.k8s.io               2024-10-21T17:55:51Z
grpcroutes.gateway.networking.k8s.io             2024-10-21T17:55:51Z
httproutes.gateway.networking.k8s.io             2024-10-21T17:55:51Z
referencegrants.gateway.networking.k8s.io        2024-10-21T17:55:51Z
tcproutes.gateway.networking.k8s.io              2024-10-21T17:55:51Z
tlsroutes.gateway.networking.k8s.io              2024-10-21T17:55:51Z
udproutes.gateway.networking.k8s.io              2024-10-21T17:55:51Z
hudson-s commented 1 month ago

Interesting, can you double check that the CRD install was successful?

To install the CRDS, Just run kubectl apply -f helm/stunner-gateway-operator/crds/ and check the output, and the available API resources and CRDs in your cluster. I have a new insall on AKS, and the output is the following:

kubectl api-resources | grep gateway
gatewayclasses                      gc                  gateway.networking.k8s.io/v1         false        GatewayClass
gateways                            gtw                 gateway.networking.k8s.io/v1         true         Gateway
grpcroutes                                              gateway.networking.k8s.io/v1alpha2   true         GRPCRoute
httproutes                                              gateway.networking.k8s.io/v1         true         HTTPRoute
referencegrants                     refgrant            gateway.networking.k8s.io/v1beta1    true         ReferenceGrant
tcproutes                                               gateway.networking.k8s.io/v1alpha2   true         TCPRoute
tlsroutes                                               gateway.networking.k8s.io/v1alpha2   true         TLSRoute
udproutes                                               gateway.networking.k8s.io/v1alpha2   true         UDPRoute
gatewayconfigs                      gwconf              stunner.l7mp.io/v1                   true         GatewayConfig

Note that we also see gatewayconfigs.stunner.l7mp.io/v1 resources in the output.

kubectl get crds | grep gateway
gatewayclasses.gateway.networking.k8s.io         2024-10-21T17:55:51Z
gatewayconfigs.stunner.l7mp.io                   2024-10-21T17:55:51Z
gateways.gateway.networking.k8s.io               2024-10-21T17:55:51Z
grpcroutes.gateway.networking.k8s.io             2024-10-21T17:55:51Z
httproutes.gateway.networking.k8s.io             2024-10-21T17:55:51Z
referencegrants.gateway.networking.k8s.io        2024-10-21T17:55:51Z
tcproutes.gateway.networking.k8s.io              2024-10-21T17:55:51Z
tlsroutes.gateway.networking.k8s.io              2024-10-21T17:55:51Z
udproutes.gateway.networking.k8s.io              2024-10-21T17:55:51Z

I just ran thekubectl apply -f helm/stunner-gateway-operator/crds/

Print as follows

# kubectl api-resources | grep gateway
gatewayclasses                     gc               gateway.networking.k8s.io/v1                  false        GatewayClass
gateways                           gtw              gateway.networking.k8s.io/v1beta1             true         Gateway
grpcroutes                                          gateway.networking.k8s.io/v1alpha2            true         GRPCRoute
httproutes                                          gateway.networking.k8s.io/v1                  true         HTTPRoute
referencegrants                    refgrant         gateway.networking.k8s.io/v1beta1             true         ReferenceGrant
tcproutes                                           gateway.networking.k8s.io/v1alpha2            true         TCPRoute
tlsroutes                                           gateway.networking.k8s.io/v1alpha2            true         TLSRoute
udproutes                                           gateway.networking.k8s.io/v1alpha2            true         UDPRoute
gatewayconfigs                     gwconf           stunner.l7mp.io/v1                            true         GatewayConfig

# kubectl get crds | grep gateway
gatewayclasses.gateway.networking.k8s.io                         2024-08-09T03:29:06Z
gatewayconfigs.stunner.l7mp.io                                   2024-10-23T12:15:16Z
gateways.gateway.networking.k8s.io                               2024-08-09T03:29:06Z
grpcroutes.gateway.networking.k8s.io                             2024-10-23T12:15:16Z
httproutes.gateway.networking.k8s.io                             2024-08-09T03:29:06Z
referencegrants.gateway.networking.k8s.io                        2024-08-09T03:29:06Z
tcproutes.gateway.networking.k8s.io                              2024-10-23T12:15:16Z
tlsroutes.gateway.networking.k8s.io                              2024-10-23T12:15:16Z
udproutes.gateway.networking.k8s.io                              2024-10-23T12:15:16Z

I am not very proficient in crd. Can you tell me what I need to do next

levaitamas commented 1 month ago

LGTM. Any update on your deployment? Does it finally work?

hudson-s commented 1 month ago

LGTM. Any update on your deployment? Does it finally work?

sorry ,no I need your help

hudson-s commented 1 month ago

hello I tried again and resolved the matter. But there is a line printed in the pod, I don't know if it has any effect

2024-10-24T02:34:01.582269558Z  INFO    renderer    No gateway-class objects found  {"event": "render: generation: 1"}
2024-10-24T02:34:01.788381301Z  INFO    udproute-controller Validating backend  {"udproute": "not found"}
2024-10-24T02:34:01.789473895Z  INFO    udproute-controller Validating backend  {"udproute": "not found"}
2024-10-24T02:34:12.852483088Z  INFO    udproute-controller Validating backend  {"udproute": "not found"}
2024-10-24T02:34:12.852514074Z  INFO    udproute-controller Validating backend  {"udproute": "not found"}
2024-10-24T02:35:45.4385956Z    INFO    node-controller failed to find node with valid external address {"reason": "End of node list reached after searching through 4 node(s)"}
2024-10-24T02:35:45.688628843Z  INFO    operator    Starting new reconcile generation   {"generation": 1, "last-acked-generation": -1}
2024-10-24T02:35:45.68866041Z   INFO    renderer    Rendering configuration {"generation": 2, "event": "render: generation: 2"}
2024-10-24T02:35:45.688665757Z  INFO    renderer    Starting dataplane render   {"mode": "managed"}
2024-10-24T02:35:45.688678733Z  INFO    renderer    No gateway-class objects found  {"event": "render: generation: 2"}
hudson-s commented 1 month ago

Q1: udproute-controller Validating backend {"udproute": "not found"} Q2 failed to find node with valid external address {"reason": "End of node list reached after searching through 4 node(s)"} I understand because I didn't leak the extranet ip node.

  1. If the Internet ip address is leaked, will the entire cluster have security problems?
  2. Do you need taint?
levaitamas commented 1 month ago

I would recommend to try a working example first. The simple-tunnel shows how to configure STUNner. Let us know whether this example works in your setup or not.

rg0now commented 4 weeks ago

Closing this as a preparation for rolling the v1 release, please reopen if you have further info.