search

wongnai / xds

Simple xDS server for Kubernetes
MIT License
104 stars 10 forks source link

gRPC xDS client stops working after a period of inactivity #11

Open rdesgroppes opened 1 month ago

rdesgroppes commented 1 month ago

We've been using your xDS Server for gRPC on Kubernetes, which suits our needs and is easy to use. However, we are experiencing problems when the channels switch to idle mode. Disabling idleness helps, but we'd now like to restore the default behavior.

I'm far from being an expert on the matter, that's why I'm kindly requesting your point of view on the issue that was originally filed against the gRPC-Java client: https://github.com/grpc/grpc-java/issues/9632.

According to https://github.com/grpc/grpc-java/issues/9632#issuecomment-1317446356:

[...] This seems like a go-control-plane bug to me. Once grpc-java sends a request with the removed resources, go-control-plane needs to honor that; it should not need any subsequent ACK/NACK. [...]

Is the xDS Server for gRPC on Kubernetes behaving like the aforementioned go-control-plane?

whs commented 1 month ago

Internally this software use go control plane, so any bugs by them will have to be fixed upstream.

Not sure if related but we do find idle bugs in grpc go, and currently our workaround is to pin grpc at a specific version that do not have revamped idle behavior

rdesgroppes commented 1 month ago

Not sure if related but we do find idle bugs in grpc go, and currently our workaround is to pin grpc at a specific version that do not have revamped idle behavior

Right, idleness is enabled by default since gRPC-Go 1.59.0. We had to create channels with an additional option (grpc.WithIdleTimeout(0)) to get around it.