Storage explorer throughput calculator values is not as the network driver stats in task manager

[mahmoudsamy27]

Preflight Checklist

• [X] I have installed the latest version of Storage Explorer.
• [X] I have checked existing resources, including the troubleshooting guide and the release notes.
• [X] I have searched for similar issues.

Storage Explorer Version

1.33.1

Regression From

No response

Architecture

i86

Storage Explorer Build Number

No response

Platform

All

OS Version

Windows 11

Bug Description

If you try to download relatively big file (10GB+) to local PC you may see that download speed shown in Mbps is jumping from 100 to 400Mbps (depends on ISP plan) but in Windows Task Manager the network driver throughput is not similar to ASE throughput calculation. It means you have a bug in speed calculation per second.

Note: Confirmed that there is no tasks consuming network bandwidth other than storage explorer

Steps to Reproduce

Download 10GB file from storage explorer and compare the throughput with network driver throughput from task manager

Actual Experience

Storage explorer calculated throughput is showing different values when compare with network driver throughput

Expected Experience

No response

Additional Context

No response

[MRayermannMSFT]

@mahmoudsamy27 do you know how network driver stats in task manager computes its value? Is it an instantaneous value? I can tell you that for Storage Explorer, AzCopy which is driving the download, is not reporting an instantaneous value. So there it is likely expected for there to be a discrepancy.

[mahmoudsamy27]

thanks for clarification, may I ask if there is work-in-progress to enhance how throughput is calculated to match wifi driver stats.

[craxal]

@mahmoudsamy27 Can you clarify just how different the values reported by Storage Explorer are from the system's values? Like what's been said before, discrepancies are expected. This is due to differences in sampling, intervals, etc. It's also difficult to get accurate values for rates that jump around a lot (such as network traffic).

[mahmoudsamy27]

There is a huge difference between the values reported by storage explorer and the system values specially on the spikes. This can be reproduced by downloading a large file 10GB+. The issue is that the values reported by storage explorer is not even near to the average.

[craxal]

@mahmoudsamy27 What numbers do you see if you run AzCopy on its own?

There are some key differences to the numbers Storage Explorer reports that would contribute to discrepancies with the system:

• Storage Explorer does not report on averages. It calculates instantaneous speed (per data batch iteration). Therefore, the system's average speeds are expected to deviate from Storage Explorer's instantaneous speeds as an operation progresses. Instantaneous speeds are useful for spotting unusual performance changes, like network slowdown.
• The speed reported by Storage Explorer is not strictly a network transfer speed. The speed is calculated by dividing the number of the bytes processed (not transferred) by the time it took for the bytes to be processed. Additional time is required by Storage Explorer and AzCopy to process and orchestrate transfers.
• AzCopy can add some additional discrepancies in speed when fiddling with its configurations. For example, if throttling it's transfer rate, AzCopy might still send large packets over the network but slow down it's processing rate to compensate. From Storage Explorer's viewpoint, it won't observe much of a difference, but the system networking stack would.

All in all, getting precise speeds is infeasible. Instantaneous speeds are a useful performance heuristic, and we do not observe any errors in how it is calculated.