search

envoyproxy / gateway

Manages Envoy Proxy as a Standalone or Kubernetes-based Application Gateway
https://gateway.envoyproxy.io
Apache License 2.0
1.53k stars 328 forks source link

Multiple routes on the same TCP listener #4150

Open S3LL1G28 opened 1 week ago

S3LL1G28 commented 1 week ago

Description:

What issue is being seen? Describe what should be happening instead of the bug, for example: Envoy should not crash, the expected value isn't returned, etc.

when I create a tcproute pointing to my tcp-gateway. I have a message when I do describe for my tcproute which is telling multiple route to listener. My tcp gateway contain many listeners .

My aim is to use one tcp gateway with only one load balanced ip and multiple ports. Then I use the only tcprout needed for my service.

here is the describe.

apiVersion: gateway.networking.k8s.io/v1alpha2                                                                                                                                                                    │
│ kind: TCPRoute                                                                                                                                                                                                    │
│ metadata:                                                                                                                                                                                                         │
│   annotations:                                                                                                                                                                                                    │
│     meta.helm.sh/release-name: whisper                                                                                                                                                                            │
│     meta.helm.sh/release-namespace: whisper                                                                                                                                                                       │
│   creationTimestamp: "2024-09-03T19:21:06Z"                                                                                                                                                                       │
│   generation: 1                                                                                                                                                                                                   │
│   labels:                                                                                                                                                                                                         │
│     app.kubernetes.io/managed-by: Helm                                                                                                                                                                            │
│   name: whisper-tcproute                                                                                                                                                                                          │
│   namespace: whisper                                                                                                                                                                                              │
│   resourceVersion: "69465268"                                                                                                                                                                                     │
│   uid: 193b1b66-cd57-4cb3-aeb4-3c814e0d4a13                                                                                                                                                                       │
│ spec:                                                                                                                                                                                                             │
│   parentRefs:                                                                                                                                                                                                     │
│   - group: gateway.networking.k8s.io                                                                                                                                                                              │
│     kind: Gateway                                                                                                                                                                                                 │
│     name: tcp-gateway                                                                                                                                                                                             │
│     namespace: gateway                                                                                                                                                                                            │
│   rules:                                                                                                                                                                                                          │
│   - backendRefs:                                                                                                                                                                                                  │
│     - group: ""                                                                                                                                                                                                   │
│       kind: Service                                                                                                                                                                                               │
│       name: whisper-common                                                                                                                                                                                        │
│       port: 10300                                                                                                                                                                                                 │
│       weight: 1                                                                                                                                                                                                   │
│ status:                                                                                                                                                                                                           │
│   parents:                                                                                                                                                                                                        │
│   - conditions:                                                                                                                                                                                                   │
│     - lastTransitionTime: "2024-09-03T19:21:39Z"                                                                                                                                                                  │
│       message: Multiple routes on the same TCP listener                                                                                                                                                           │
│       observedGeneration: 1                                                                                                                                                                                       │
│       reason: UnsupportedValue                                                                                                                                                                                    │
│       status: "False"                                                                                                                                                                                             │
│       type: Accepted                                                                                                                                                                                              │
│     - lastTransitionTime: "2024-09-03T19:21:39Z"                                                                                                                                                                  │
│       message: Resolved all the Object references for the Route                                                                                                                                                   │
│       observedGeneration: 1                                                                                                                                                                                       │
│       reason: ResolvedRefs                                                                                                                                                                                        │
│       status: "True"                                                                                                                                                                                              │
│       type: ResolvedRefs                                                                                                                                                                                          │
│     controllerName: gateway.envoyproxy.io/gatewayclass-controller                                                                                                                                                 │
│     parentRef:                                                                                                                                                                                                    │
│       group: gateway.networking.k8s.io                                                                                                                                                                            │
│       kind: Gateway                                                                                                                                                                                               │
│       name: tcp-gateway                                                                                                                                                                                           │
│       namespace: gateway

my tcproute-gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"gateway.networking.k8s.io/v1","kind":"Gateway","metadata":{"annotations":{},"name":"tcp-gateway","namespace":"gateway"},"spec":{"addresses":[{"type":"IPAddress","value":"10.10.1.160"}],"gatewayClassName":"eg","listeners":[{"allowedRoutes":{"kinds":[{"kind":"TCPRoute"}],"namespaces":{"from":"All"}},"name":"piper","port":10200,"protocol":"TCP"},{"allowedRoutes":{"kinds":[{"kind":"TCPRoute"}],"namespaces":{"from":"All"}},"name":"whisper","port":10300,"protocol":"TCP"},{"allowedRoutes":{"kinds":[{"kind":"TCPRoute"}],"namespaces":{"from":"All"}},"name":"rhasspy","port":12101,"protocol":"TCP"},{"allowedRoutes":{"kinds":[{"kind":"TCPRoute"}],"namespaces":{"from":"All"}},"name":"zwavejs","port":3000,"protocol":"TCP"},{"allowedRoutes":{"kinds":[{"kind":"TCPRoute"}],"namespaces":{"from":"All"}},"name":"ocpp","port":9000,"protocol":"TCP"}]}}
  creationTimestamp: "2024-08-29T13:53:51Z"
  generation: 5
  name: tcp-gateway
  namespace: gateway
  resourceVersion: "69465261"
  uid: 6eaccd1d-b033-4144-94c6-be7f0c7b8300
spec:
  addresses:
  - type: IPAddress
    value: 10.10.1.160
  gatewayClassName: eg
  listeners:
  - allowedRoutes:
      kinds:
      - group: gateway.networking.k8s.io
        kind: TCPRoute
      namespaces:
        from: All
    name: piper
    port: 10200
    protocol: TCP
  - allowedRoutes:
      kinds:
      - group: gateway.networking.k8s.io
        kind: TCPRoute
      namespaces:
        from: All
    name: whisper
    port: 10300
    protocol: TCP
  - allowedRoutes:
      kinds:
      - group: gateway.networking.k8s.io
        kind: TCPRoute
      namespaces:
        from: All
    name: rhasspy
    port: 12101
    protocol: TCP
  - allowedRoutes:
      kinds:
      - group: gateway.networking.k8s.io
        kind: TCPRoute
      namespaces:
        from: All
    name: zwavejs
    port: 3000
    protocol: TCP
  - allowedRoutes:
      kinds:
      - group: gateway.networking.k8s.io
        kind: TCPRoute
      namespaces:
        from: All
    name: ocpp
    port: 9000
    protocol: TCP
status:
  addresses:
  - type: IPAddress
    value: 10.10.1.160
  conditions:
  - lastTransitionTime: "2024-09-03T19:21:39Z"
    message: The Gateway has been scheduled by Envoy Gateway
    observedGeneration: 5
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2024-09-03T19:21:39Z"
    message: Address assigned to the Gateway, 1/1 envoy Deployment replicas available
    observedGeneration: 5
    reason: Programmed
    status: "True"
    type: Programmed
  listeners:
  - attachedRoutes: 2
    conditions:
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Sending translated listener configuration to the data plane
      observedGeneration: 5
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener has been successfully translated
      observedGeneration: 5
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener references have been resolved
      observedGeneration: 5
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    name: piper
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: TCPRoute
  - attachedRoutes: 2
    conditions:
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Sending translated listener configuration to the data plane
      observedGeneration: 5
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener has been successfully translated
      observedGeneration: 5
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener references have been resolved
      observedGeneration: 5
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    name: whisper
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: TCPRoute
  - attachedRoutes: 2
    conditions:
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Sending translated listener configuration to the data plane
      observedGeneration: 5
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener has been successfully translated
      observedGeneration: 5
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener references have been resolved
      observedGeneration: 5
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    name: rhasspy
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: TCPRoute
  - attachedRoutes: 2
    conditions:
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Sending translated listener configuration to the data plane
      observedGeneration: 5
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener has been successfully translated
      observedGeneration: 5
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener references have been resolved
      observedGeneration: 5
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    name: zwavejs
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: TCPRoute
  - attachedRoutes: 2
    conditions:
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Sending translated listener configuration to the data plane
      observedGeneration: 5
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener has been successfully translated
      observedGeneration: 5
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-09-03T19:21:39Z"
      message: Listener references have been resolved
      observedGeneration: 5
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    name: ocpp
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: TCPRoute

Repro steps:

Include sample requests, environment, etc. All data and inputs required to reproduce the bug.

Note: If there are privacy concerns, sanitize the data prior to sharing.

Environment:

Include the environment like gateway version, envoy version and so on.

eg      envoy-gateway-system    1               2024-08-18 18:12:28.978948066 +0200 CEST        deployed        gateway-helm-v1.1.0     v1.1.0

Logs:

Include the access logs and the Envoy logs.

my envoy gateway tcp logs:

envoy [2024-09-03 19:20:48.080][1][warning][config] [source/extensions/config_subscription/grpc/delta_subscription_state.cc:269] delta config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: │
│ envoy gateway/tcp-gateway/ocpp: error adding listener '0.0.0.0:9000': no filter chains specified                                                                                                                  │
│ envoy gateway/tcp-gateway/piper: error adding listener '0.0.0.0:10200': no filter chains specified                                                                                                                │
│ envoy gateway/tcp-gateway/whisper: error adding listener '0.0.0.0:10300': no filter chains specified                                                                                                              │
│ envoy gateway/tcp-gateway/rhasspy: error adding listener '0.0.0.0:12101': no filter chains specified                                                                                                              │
│ envoy                                                                                                                                                                                                             │
│ envoy [2024-09-03 19:20:48.081][1][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:138] gRPC config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Er │
│ envoy gateway/tcp-gateway/ocpp: error adding listener '0.0.0.0:9000': no filter chains specified                                                                                                                  │
│ envoy gateway/tcp-gateway/piper: error adding listener '0.0.0.0:10200': no filter chains specified                                                                                                                │
│ envoy gateway/tcp-gateway/whisper: error adding listener '0.0.0.0:10300': no filter chains specified                                                                                                              │
│ envoy gateway/tcp-gateway/rhasspy: error adding listener '0.0.0.0:12101': no filter chains specified

I hope you'll have all information to see if it's a bug or a misconfiguration.

Thx.

arkodg commented 1 week ago

this error Multiple routes on the same TCP listener indicates that you have may have multiple TCPRoutes attached to the same gateway or gateway listener, which is not possible to support for TCP.

S3LL1G28 commented 1 week ago

I don't understand . In this URL WE saw one TCP gateway with multiples tcproutes. https://gateway.envoyproxy.io/docs/tasks/traffic/tcp-routing/

arkodg commented 1 week ago

I don't understand . In this URL WE saw one TCP gateway with multiples tcproutes. https://gateway.envoyproxy.io/docs/tasks/traffic/tcp-routing/

in the above example, each route is linked with a different gateway listener using the sectionName field in the parentRefs field

S3LL1G28 commented 1 week ago

Ok so what's wrong in my conf ? iI have to add sectionName in each tcproute ? Does my tcpgateway is well setup?

S3LL1G28 commented 1 week ago

Pff i'm stupid. Effectively it was sectionname on each tcproute which was missing. Everything IS ok now. Thanks you for opening my eyes and my apologies to disturb you. And great job for this project .