Issue with the "Load Balancing" doc RE: in-region loadbalancing

[swilson-fly]

I found an issue with this document.

Title: Load Balancing Location: https://fly.io/docs/reference/load-balancing/ Source: https://github.com/superfly/docs/blob/main/reference/load-balancing.html.md

Describe the issue

The Proxy as currently configured will avoid routing requests outside a given region until all machines in that region are over their hard_limit. The current examples state that we'll start routing to nearby regions once all machines in them are over the soft limit. This is incorrect!

This example section needs updating:

In this contrived example, all of the users are currently in Amsterdam, so the traffic is arriving at one of the Fly.io edges in Amsterdam. Here’s what happens as the number of concurrent HTTP requests from users in Amsterdam increases:

60 concurrent requests: Requests are divided evenly between the 3 Machines in the ams region. Closeness of the worker and edge will determine which of the 3 Machines each request goes to.
61+ concurrent requests: 60 of those requests will be sent to the 3 Machines in the ams region and the rest will be sent to the closest Machines in other regions.
200+ concurrent requests: All Machines are at their soft_limit, so Fly Proxy will start routing requests to the ams Machines again. For example, the 201st concurrent request will go to a Machine in the ams region that is currently at its soft_limit.
250 concurrent requests: All Machines are at their hard_limit. The 251st concurrent request will get queued by Fly Proxy until a Machine is below its hard_limit.

Addtional info

Doublechecking with the networking team to ensure that the change from what's documented was intentional. Once that's confirmed I can make a PR to update this

[andie787]

Sounds good! Thanks for this!

[AshitaGK]

Any follow up on this? Shouldn't Fly Proxy deprioritize/queue requests when all the Machines are above their soft_limit?

According to the documentation for Dynamic Request Routing, intro paragraph mentions that sometimes it’s useful or necessary to route requests to other regions, other apps or both. Kindly clarify.