search

microsoft / reverse-proxy

A toolkit for developing high-performance HTTP reverse proxy applications.
https://microsoft.github.io/reverse-proxy
MIT License
8.46k stars 832 forks source link

Slow Start Mode #2430

Open andhesky opened 6 months ago

andhesky commented 6 months ago

What should we add or change to make your life better?

Slow start mode is a way to give newly started applications an ability to warm up before taking their full share of traffic.

On startup, web applications may need to fill caches, establish connections to upstream services, and create threads in their thread pool. This warmup period can sometimes be simulated before reporting as ready, but the best way to guarantee that the applications are ready to take the full traffic load is to give them a small portion of the traffic and ramp it up over a few seconds or minutes.

This has been implemented in Envoy as well as some of the K8s ingress providers based on Envoy like Istio warmupDurationSecs and Contour Slow Start Mode

Why is this important to you?

Without this feature, applications see latency spikes and availability drops any time they need to scale up. In a cloud environment with autoscale enabled, there may be dozens of scale up operations per day negatively impacting end users.

Tratcher commented 6 months ago

There may be some functional/conceptual overlap with Degraded health state. https://github.com/microsoft/reverse-proxy/issues/2071