projectcontour / contour

Contour is a Kubernetes ingress controller using Envoy proxy.
https://projectcontour.io
Apache License 2.0
3.67k stars 668 forks source link

Stacking LDS updates cause non-deterministic Envoy memory usage #1635

Open phylake opened 4 years ago

phylake commented 4 years ago

We regularly see 3+ total_listeners_draining in Envoy stats. Each of these has a full set of filter chain matches and consume quite a bit of memory for our 200+ FQDNs

Regardless of any changes in Envoy (see https://github.com/envoyproxy/envoy/issues/7923), Contour should ensure LDS doesn't stack up in Envoy. There should be a guarantee of, for example, 2 ingress_http (at most 1 draining) + 2 ingress_https (at most 1 draining) listeners as an upper bound

davecheney commented 4 years ago

Thank you for raising this issue.

I think the best solution to this is FDS which we’re waiting on. Maybe it’ll come in envoy 1.12.

With that said I understand the issues that frequent LDS update cause in your environment and I will continue to work to reduce them. I don’t have a concrete eta on when this will happen or what it will involve but 499 and related issues remain a high priority for me.

On 5 Oct 2019, at 02:13, Brandon Cook notifications@github.com wrote:

 We regularly see 3+ total_listeners_draining in Envoy stats. Each of these has a full set of filter chain matches and consume quite a bit of memory for our 200+ FQDNs

Regardless of any changes in Envoy (see envoyproxy/envoy#7923), Contour should Ensure LDS doesn't stack up in Envoy. There should be a guarantee of, for example, 2 ingress_http (at most 1 draining) + 2 ingress_https (at most 1 draining) listeners as an upper bound

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

github-actions[bot] commented 4 days ago

The Contour project currently lacks enough contributors to adequately respond to all Issues.

This bot triages Issues according to the following rules:

You can:

Please send feedback to the #contour channel in the Kubernetes Slack