easwars
commented
11 months ago LEDS is something that we would really want gRPC to support in the long run, but cannot commit to working on it now, given our existing priorities and commitments. LEDS is built on top of the incremental xDS protocol variant (which gRPC does not support at this point in time). It also requires support for glob collections (which are described in https://github.com/cncf/xds/blob/main/proposals/TP1-xds-transport-next.md#glob). It is a fairly big project, and the first step for anything as big as this would be a gRFC in the grpc-proposal repository.
As mentioned above, we cannot commit to this right now. But you are welcome to write a gRFC, which we can help guide and review. Once the gRFC is approved, you are welcome to contribute an implementation too, and we would be very happy to review the same.
Please see the FAQ in our main README.md before submitting your issue.
Use case(s) - what problem will this feature solve?
Our large service cluster, comprising over 3,000 nodes, is causing significant memory pressure/processing overhead on the clients due to the absence of LEDS or a patch mechanism in the existing EDS. Given the immediate impact on real-world production systems, prioritizing this work would have a high impact and would be greatly appreciated:)
Proposed Solution
LEDS enables a client to subscribe to individual nodes within a service cluster instead of the entire cluster, allowing the xDS server to send updates on individual servers rather than the entire 3000+ node cluster.
Envoy already supports LEDS in delta mode. It would be immensely helpful and have significant impacts if gRPC followed the same path. As mentioned by @markdroth, "gRPC does not currently support either the incremental protocol variants nor LEDS, but I would very much like to see us support those things in the long run. I think it's really just a question of when we encounter a concrete use case that requires that functionality, which would cause us to prioritize the work."
Alternatives Considered
We initially considered server-side subsetting to be fully compatible with the basic xDS (EDS) protocol. However, implementing server-side subsetting presents challenges due to the constant dynamics of service redeployments. The need for dynamic subsetting arises when a partial set of servers becomes unavailable without affecting the client. Additionally, server-side subsetting can interfere with client-side subsetting in our legacy clients.
It appears that other proposals aimed at improving the scalability of large clusters (e.g., https://github.com/envoyproxy/envoy/issues/9455) have not gained as much traction as LEDS.
Additional Context
this feature request was discussed and recommended by @markdroth in https://github.com/envoyproxy/envoy/issues/31071