christophermaier
commented
4 years ago Ideally, I think I'd want to target package downloads for throttling first, as opposed to all Supervisor network activity, which would include gossip. I'd certainly want to treat those two classes of activity separately; it would be bad for gossip activity to get throttled relative to download activity, since the former is vitally important to the operation of the system as a whole.
One idea I had is to implement a kind of wrapper around our HTTP client that can inspect the progress of any HTTP request futures that are in flight and keep them throttled as necessary to stay within some configurable bandwidth limit. The Tower project has some interesting composable middleware functionality that is at least architecturally along these lines. They have a few interesting implementations that do rate limiting kinds of things, but it appears to be more of the request-per-second kind, rather than Mbps-per-second, which seems to be what we'd want. I haven't yet seen an obvious way (outside of patching hyper or reqwest) to get insight into the amount of bytes a request has received yet, though.
I think there are some other easier changes we can make that will reduce our network usage that we should tackle before we dig into this. It could still be useful functionality to implement at some point, though, so we'll keep this card open for now to capture additional usecase information, implementation ideas, etc.
stale[bot]
Depending on the environment in which a Supervisor is running, it may be desirable to have it be able to rate-limit its network activity. This can be of particular concern with customers running Supervisors in low-bandwidth locations, where the Supervisor must share that bandwidth with other important uses. Network QOS configuration is a solution, but may not always be available.
Though the Supervisor uses the network for gossip, the more pressing concern appears to surround the downloading of new packages for service and supervisor updates. As such, the Supervisor would need some way to either limit each of these individually, or limit them all collectively.