VHDS not working for me

[rettuna]

I am trying to configure the virtual hosts on envoy using VHDS. This is probably me, but I can't seem to get it to work. I am building on the example from the go-control-plane repo.

    snap, _ := cache.NewSnapshot(version,
        map[string][]types.Resource{
            resource.ListenerType:    listens,
            resource.RouteType:       routeConfigs,
            resource.VirtualHostType: virtualHosts,
        })

My configs look like this.

Route config:

{
        "name": "myrouteconfig0",
        "vhds": {
            "config_source": {
                "api_config_source": {
                    "api_type": "DELTA_GRPC",
                    "transport_api_version": "V3",
                    "grpc_services": [
                        {
                            "envoy_grpc": {
                                "cluster_name": "xds_cluster"
                            }
                        }
                    ]
                }
            }
        }
    }

Virtual host:

    {
        "name": "myrouteconfig0/vh1",
        "domains": [
            "*"
        ],
        "routes": [
            {
                "match": {
                    "path": "/foo1"
                },
                "direct_response": {
                    "status": 200,
                    "body": {
                        "inline_string": "Hello"
                    }
                }
            }
        ]
    }

Another issue I hit was trying to set "config_source":{"ads":{}}, gives me the error err:vhds: only 'DELTA_GRPC' is supported as an api_type. How ever I can specify this for RDS in the same snapshot.

Any help appreciated, thanks!

[valerian-roche]

Hey

Can you provide more details on the setup you have? The envoy configuration seems functional, but the virtual host configuration seems invalid to me. Is "vh1" the host in the inbound request? Overall can you provide logs of the query received by your management server from envoy, as well as details on your setup (linear or snapshot cache for instance)?

[rettuna]

Thanks @valerian-roche. Here are a few more details.

I have only an ADS only server running with snapshot cache. I am using the delta server package directly.

deltaSrv = delta.NewServer(ctx, s.cache, s)

//With handlers passing new streams to the DeltaStreamHandler
func (s *server) DeltaAggregatedResources(stream discovery.AggregatedDiscoveryService_DeltaAggregatedResourcesServer) error {
    return s.deltaSrv.DeltaStreamHandler(stream, resource.AnyType)
}

func (s *server) StreamAggregatedResources(stream discovery.AggregatedDiscoveryService_StreamAggregatedResourcesServer) error {
    return errors.New("not supported")
}

When I configure below I get the error ".....RouteConfiguration rejected: vhds: only 'DELTA_GRPC' is supported as an api_type." However the same thing works in HCM for rds.

    {
        "name": "myrouteconfig0/vh1",
        "vhds": {
            "config_source": {
                "ads": {}
            }
        }

If I change it to below, then envoy complains - ......rce/extensions/config_subscription/grpc/grpc_stream.h:155] DeltaVirtualHosts gRPC config stream to xds_cluster closed: 12, unknown service envoy.service.route.v3.VirtualHostDiscoveryService

    {
        "name": "myrouteconfig0/vh1",
        "vhds": {
            "config_source": {

                "api_config_source": {
                    "api_type": "DELTA_GRPC",
                    "transport_api_version": "V3",
                    "grpc_services": [
                        {
                            "envoy_grpc": {
                                "cluster_name": "xds_cluster"
                            }
                        }
                    ]
                }
            }
        }

[valerian-roche]

From the envoy error it seems the issue is not on the go-control-plane but on the grpc server not registering the vhds service in its registration (akin to https://github.com/envoyproxy/go-control-plane/blob/main/examples/dyplomat/main.go#L50, but using https://github.com/envoyproxy/go-control-plane/blob/main/envoy/service/route/v3/rds_grpc.pb.go#L321).

[rettuna]

I see. I do register discovery.RegisterAggregatedDiscoveryServiceServer(s.grpc, s) the ADS service with the grpc server, which I assumed should serve Virtual host resource as well(because it works for listeners and route configuration)

Based on your comment looks like setting the api_type to "DELTA_GRPC" requires a incremental XDS server to be listening for virtual host requests(bullet point 2 per documentation here. Is there any way to ask envoy to use ADS instead of incremental XDS for virtual hosts?

[valerian-roche]

I do not know the internals of envoy which pushed them to not allow ADS as configuration, even if ADS itself is configured to use DELTA_GRPC. Might be related to this piece of code not having access to the ADS client multiplexer, or being able to know if this uses delta or not. Using DELTA_GRPC instead of ADS here should only require you to register the proper service, as the go-control-plane will properly serve the objects behind the scene. If you do some extensions through callbacks you'd have to also support the requests in this case, but they are mostly the same (apart from the type being set at a different place).

[rettuna]

Thanks a lot @valerian-roche. I appreciate your help in resolving this issue.