search

envoyproxy / java-control-plane

Java implementation of an Envoy gRPC control plane
Apache License 2.0
293 stars 136 forks source link

Instance stays when priority is changed #96

Open jakubdyszkiewicz opened 5 years ago

jakubdyszkiewicz commented 5 years ago

When I set the snapshot with the instance in locality dc1 with priority 0 and then set another one with instance in locality dc1 with priority 1, but without the first one, there are two instances in Envoy.

I've written following test https://github.com/jakubdyszkiewicz/java-control-plane/commit/39b76bc71fe48ab8f727d66714ef3f3f427dc039

But then it turned out that the problem seems to be even more general. When I set a snapshot with endpoint with priority 0, then set another one with priority 1, but without the first one - both stays in Envoy. https://github.com/jakubdyszkiewicz/java-control-plane/blob/a2a6b4b5498fa1de734ed76aafcdf768067f525a/server/src/test/java/io/envoyproxy/controlplane/server/MoveInstanceToAnotherLocalityTest.java

Not sure whether this is an issue with java-control-plane or Envoy itself or this is just an expected behaviour.

I've seen issues in Envoy that may be connected with this https://github.com/envoyproxy/envoy/issues/3327 https://github.com/envoyproxy/envoy/issues/3312

@snowp it would be helpful if you could take a look at this, because you fixed above issues. Thank you.

snowp commented 5 years ago

This sounds like the intended Envoy behavior when an active health checker is present (see https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/service_discovery#on-eventually-consistent-service-discovery), but it shouldn't really come into play without a health checker.

Maybe try setting drain_connections_on_host_removal on the cluster to see if that fixes it? In any case, this seems like an Envoy thing more than a control plane issue.