This commit implements stateful client-side
load balancing. A Client and Enclave now
contain an internal loadBalancer which sends
requests to a list of endpoints.
The load balancer marks an endpoint as offline
when an endpoint fails to respond, e.g. due to
a network error or 5xx response code. Offline
endpoints no longer receive requests.
After some time period (currently 5m), the load
balancer marks offline endpoints as online to
check if it has become available. If the endpoint
still fails to respond it gets marked offline again for another 5m.
This process ensures that offline failing endpoints don't receive traffic while at the same time endpoints can become available again.
This commit implements stateful client-side load balancing. A
ClientandEnclavenow contain an internalloadBalancerwhich sends requests to a list of endpoints.The load balancer marks an endpoint as offline when an endpoint fails to respond, e.g. due to a network error or 5xx response code. Offline endpoints no longer receive requests.
After some time period (currently 5m), the load balancer marks offline endpoints as online to check if it has become available. If the endpoint still fails to respond it gets marked offline again for another 5m.
This process ensures that offline failing endpoints don't receive traffic while at the same time endpoints can become available again.