Quality of a bandwidth estimate

[vasilvv]

There might be utility in exposing some kind of "quality" indication of a bandwidth estimate, indicating how close it is to the "real" bandwidth of the channel.

There are few signals that congestion controller could expose as an API that I can think of:

• Has there ever been a bandwidth sample that is not application-limited?
• Is the connection out of slow start?

cc @ianswett

[ianswett]

There definitely are differences in the accuracy of bandwidth signals and Victor pointed out a few.

None of these are perfect, but possibly it'd be useful to know if the bandwidth was estimated to be the full bandwidth or a minimum of what's been observed?

[wilaw]

Responses received at IETF #118 WebTransport session.

Not much support or feedback for the suggested signals of application limited, slows start and max/min/avg. Some consistent feedback received around providing rate estimates over different timeframes. Also suggestion to align with work being done in WebRTC in this regard.

[jan-ivar]

• Has there ever been a bandwidth sample that is not application-limited?
• Is the connection out of slow start?

I'm no CC expert but these two properties of the UA's congestion algorithm seem like signals an application might put to good use. e.g. as another stat. cc @jesup thoughts?

[wilaw]

For API surface, we can add directly to WebTransportConnectionStats dictionary, or if we anticipate many additional sendRate qualifiers (such as min/avg/max) then we could create a new EstimatedSendRateStats dictionary to encapsulate them

partial dictionary WebTransportConnectionStats {
  unsigned long long estimatedSendRate;
  boolean estimatedSendRateApplicationLimited;
  boolean estimatedSendRateInSlowStart;
};

or

partial dictionary WebTransportConnectionStats {
 EstimatedSendRateStats rateEstimate;
};

dictionary EstimatedSendRateStats {
   unsigned long long estimatedSendRate;
   boolean estimatedSendRateApplicationLimited;
   boolean estimatedSendRateInSlowStart;
};

[jan-ivar]

Meeting:

• An indicator whether estimate is reliable vs unreliable (or stable vs unstable)
• What would an app do differently?
• ApplicationLimited = you might have more
• What tier do I pick on startup?
• information to speed up the app's convergence to a good value. Should I try more?
• Whether I can push more depends on the cc algorithm.
• might be just shy of the limit (a limitation of boolean)
• "Does UA think we should try"?
• Need more feedback from use cases we want to service
• E.g. sending simulcast, can I add a layer?
• (for comparison libwebrtc starts send probing packets)
• OR (idea) could let application send probes on a separate stream
• Could send datagrams with zero timeout?

[jan-ivar]

"applicationLimited" sounds like it's from the viewpoint of implementers. Maybe invert it to "networkLimited" to match the pov of web developers?

[martinthomson]

Application limited is a congestion control artifice that expresses a lack of data to send. The effect is that the network layer cannot drive the network to capacity, which means that the congestion controller cannot be confident that it is increasing throughput, hence it doesn't increase its estimate of path throughput. So "data limited" or similar would be accurate, or even "throughput rate increases capped" would reflect the effect of the switch, but this is not reflective of the state of the network.

[jan-ivar]

Meeting:

• Two factors: experienced max throughout and experience from congestion
• network limited = UA delayed some data, can change over time e.g. from cross-traffic in the next 10 ms
• packet loss from non-congestion control causes
• @jesup to ask Lars
• Feedback: apps care about short-term (next 10 ms) and longer term. Apps may want to supply a hint? one rtt-ish? smoothed? App could do its own smoothing of the short-term value.
• question about size is about knowing the congestion window? E.g. deciding a keyframe that fits within (not likely from this api)
• what is the size of the frame I'll be able to send 30 ms from now?
• Can we get some use cases where an application needs to make a decision and what rational inputs this would require? E.g. send high-bandwidth or low-bandwidth data, a 30 ms decision

[ianswett]

Application limited is a congestion control artifice that expresses a lack of data to send. The effect is that the network layer cannot drive the network to capacity, which means that the congestion controller cannot be confident that it is increasing throughput, hence it doesn't increase its estimate of path throughput. So "data limited" or similar would be accurate, or even "throughput rate increases capped" would reflect the effect of the switch, but this is not reflective of the state of the network.

"Data limited" works for me. Or we could have a term indicating what it means, like "Full bandwidth measured"?

[aboba]

My understanding is that "data limited" does not mean that the application can send (much) more. For example, in WebRTC, if a simulcast layer is dropped due to loss, and available bandwidth recovers, the BWE will be "data limited" unless the CC algorithm (e.g. Google CC) has a mechanism to "probe" (e.g. to determine what bandwidth availability really is). This is not something that the Application can do on its own; if it sent its own "probes" they would potentially be queued and not treated the same as equivalents originated by the CC algorithm on its own.

That leaves me wondering what an application can do with "data limited" except perhaps to wait for the CC algorithm to leave that regime and provide a better (and higher) estimate.

[jan-ivar]

My understanding is that "data limited" does not mean that the application can send (much) more. For example, in WebRTC, if a simulcast layer is dropped due to loss, and available bandwidth recovers, the BWE will be "data limited" ...

Unlike WebTransport, WebRTC makes decisions on behalf of the sending app, altering what is sent. A multi-chef problem.

Dropping a simulcast layer would be an application decision in WebTransport. So I'd say does mean it can send more. Unless I'm missing something.

Running with "data limited" to explore API, would it be a boolean? E.g.

const {estimatedSendRate, rateIsDataLimited} = await wt.getStats();

[jan-ivar]

Meeting:

• could apps probe and find this information anyway?
• would it be useful for an app to know:
  • is estimate over last 30 seconds?
  • am I in slow start?
• @vasilvv to take another look and comment on next steps

[jan-ivar]

Meeting:

• estimate is more accurate when we hit the ceiling
• vs we don't have a clue, try more
• applications actions: slow down, speed up, or stay the same, or e.g. choose to defer to send a keyframe
• two choices:
  1. going in the direction of exposing all the details, e.g. multiplicative decrease, additive increase, slow start etc.
  2. expose a boolean that says we have hit (are hitting) the ceiling
• Meeting participants leaning toward 2
• We could add 1 later.

[wilaw]

To prepare for future fields, we may want to encapsulate all sendRate specific data within their own object

const stats = await wt.getStats();
const estimate = stats.sendRate.estimatedRate;
const limited  = stats.sendRate.isDataLimited;
const slow = stats.sendRate.inSlowStart;
const mIncrease = stats.sendRate.inMultiplicativeDecrease;

We could add isDataLimited for now and then add others in the future if strong use-cases warrant.

[jan-ivar]

Meeting:

• Some anticipation of future fields
• Not too late to rename stats.estimatedSendRate to stats.sendRate.estimatedRate
• Some takers for isDataLimited
• Things like inSlowStart might not be relevant in every CC algorithm (absent or false?)
• stats.estimatedSendRate is implemented
• The others are not necessarily a property of sendRate, orthogonal
• How about stats.send.estimatedRate? Ship may have sailed (bytesSent etc)
• More info needed on isDataLimited?
• Something like inSlowStart seems easy
• Try PR for stats.isSendDataLimited and stats.inSlowStart?

[jan-ivar]

Meeting:

• A choice about whether we want new stats when estimatedSendRate is null: together in a single nullable object, or separate stats
• isReceiverLimited = true means backpressure. Don't probe further, but it's not the network's fault.
• Update PR to go back to stats.sendRate?.estimatedRate and add isReceiverLimited