Handle Server Reflection for multiple gRPC service

[mazzy89]

Description: Currently if multiple gRPC services are registered, Server Reflection will overlap.

By today

a user needs to be define a reflection match within a GRPCRoute for a specific hostname/authority and then rely on the authority to demux to the right backend

Relevant Links: https://envoyproxy.slack.com/archives/C03E6NHLESV/p1692986469248179

[arkodg]

Hi @mazzy89 can you share a sample Gateway API config (Gateway + GRPCRoute) and outline your expectations using a client like grpccurl list ?

thinking out loud, one design pattern could be to create 1 GRPCRoute that matches on grpc.reflection.v1alpha.ServerReflection and mirrors the request to all backends using the mirror filter 😆 , something like

kind: GRPCRoute
metadata:
  name: reflection-route
  labels:
    example: grpc-routing
spec:
  parentRefs:
    - name: example-gateway
  hostnames:
    - "*"
  rules:
    - matches:
      - method:
          method: ServerReflectionInfo
          service: grpc.reflection.v1alpha.ServerReflection
      filters:
      - type: RequestMirror
        requestMirror:
          backendRef:
            kind: Service
            name: grpc-svc-1
            port: 3000
      - type: RequestMirror
        requestMirror:
          backendRef:
            kind: Service
            name: grpc-svc-2
            port: 3000
      - type: RequestMirror
        requestMirror:
          backendRef:
            kind: Service
            name: grpc-svc-3
            port: 3000
      backendRefs:
        - group: ""
          kind: Service
          name: grpc-svc-4
          port: 9000
          weight: 1

[arkodg]

nvm please ignore above example, envoy ignores responses from mirrors https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/route/v3/route_components.proto#config-route-v3-routeaction-requestmirrorpolicy

[mazzy89]

I will post a configuration of how it should look like.

[mazzy89]

Let's assume to have two gRPC services behind EnvoyGateway foo and bar.

apiVersion: gateway.networking.k8s.io/v1alpha2
kind: GRPCRoute
metadata:
  name: foo
spec:
  parentRefs:
    - name: example-gateway
  hostnames:
    - "grpc-example.com"
  rules:
    - matches:
      - method:
          method: ServerReflectionInfo
          service: grpc.reflection.v1alpha.ServerReflection
      backendRefs:
        - group: ""
          kind: Service
          name: foo
          port: 9000
          weight: 1
----
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: GRPCRoute
metadata:
  name: bar
spec:
  parentRefs:
    - name: example-gateway
  hostnames:
    - "grpc-example.com"
  rules:
    - matches:
      - method:
          method: ServerReflectionInfo
          service: grpc.reflection.v1alpha.ServerReflection
      backendRefs:
        - group: ""
          kind: Service
          name: bar
          port: 9090
          weight: 1

when I would run grpcurl grpc-example.com:443 list or grpcurl grpc-example.com:443 describe I would like to see the list of all the services and methods.

Currently this produce the list of methods of the first registered services. The others are overlapped.

[arkodg]

above request is not possible today, because the current Gateway API semantics are 1 request is routed to 1 backendRef based on route matches. For case above the matches are identical (same hostname, same method, same service)

to make above request possible 1 request would need to be converted into 2 intermediate requests

as a workaround can you use different hostnames to talk to each backendRef bar.grpc-example.com & foo.grpc-example.com ?

[mazzy89]

Even having different matches it won't help. Server reflection will get overlapped. Actually the workaround I made was to create another service and expose the reflection of all the service through it and register the server reflection in the grpcroute of that service.

[mazzy89]

Also in my opinion this reflection should be treated internally by envoyproxy differently. Eveytime a new grpc service is registered his reflection would be aggregate together with all the other existing reflection.