search

m-lab / ndt-server

docker native ndt5 and ndt7 server with prometheus integration
https://www.measurementlab.net/
Apache License 2.0
99 stars 40 forks source link

ndt7: AppInfo should be mandatory for counterflow messages during upload tests #358

Open robertodauria opened 2 years ago

robertodauria commented 2 years ago

This is a proposal to change the ndt7 protocol specification. It has been discussed with @bassosimone (original author of the ndt7 spec) at https://github.com/m-lab/ndt7-client-go/pull/75.

Currently, both TCPInfo and AppInfo are optional fields for a Measurement object. When the client is uploading, the sender-side rate is overestimated due to buffers and data still "in transit" at the end of the measurement period. To get an accurate rate, the client must rely on Measurement messages periodically sent by the server. However, an ndt7 server today can be compliant without implementing TCPInfo nor AppInfo, which would make the clients blind to how much data has been actually received by the server and unable to show a meaningful rate to the user.

ndt-server currently sends TCPInfo only, which provides network-level byte count, and that's what our clients use. However, there is a significant asymmetry in what clients show: when downloading, the rate is computed at the application level -- (size of WS payloads / elapsed time) while when uploading the rate is computed at the network level -- (bytes received by the server / elapsed time). Clients cannot generally access TCPInfo metrics (most of our measurements come from Javascript-based implementations) so using a network-level rate for the download is not an option, but we can be consistent by showing application-level measurements in both tests.

Making AppInfo mandatory in the spec for counterflow messages (and adding it to ndt-server) would solve both these problems, guaranteeing that the clients get at least one source of accurate measurements and that we're not mixing apples and oranges by providing rates computed at different levels of the OSI model.

mattmathis commented 2 years ago

It should probably specify a frequency (range) as well.

stephen-soltesz commented 1 year ago

My understanding is that the use case we are seeking to address is archival storage of goodput measurements.

Please be precise about the source of the following information:

This summary may be helpful for updates to the ndt7 spec or to inform the MSAK design.

I know not all are available. So, which will we regard as authoritative, advisory, part of presentation only or part of archival data? Can we meet the use case? If so, what will we change? If not, what do we need to do instead?

robertodauria commented 1 year ago

My understanding is that the use case we are seeking to address is archival storage of goodput measurements.

The original intention was to provide the client with a reliable source of application-level goodput on upload tests, which we currently lack.

* Client visible goodput, upload and download
* Server visible goodput, upload and download
* Client visible network throughput, upload and download
* Server visible network throughput, upload and download