Restore and HTTP timeout (probably due to TCP parallel connections being throttled)

xen2 commented 5 years ago

Please read the following information before posting the issue.

Details about Problem

NuGet product used: NuGet client API

NuGet version (x.x.x.xxx): 5.0

OS version: win10

Detailed repro steps so we can see the same problem

Only happen on slow connections:

Install Xenko Launcher (https://xenko.com/download)
Install latest version of Xenko Note: Xenko is quite big, around 60 packages with a total size of 300 MB.
Some people (with slow connections) have timeout issues: https://github.com/xenko3d/xenko/issues/310

Other suggested things

My best guess on what's happening: Some ISP throttle TCP connections: even though NuGet open 10 TCP connections, only 2 will actively progress and the other will be put on hold until some other finishes. Given a slow enough download rate, 2 running download could last more than 100 second and will result in other pending connections to be considered timeout by NuGet (note: the timeout is not at HTTP level but just NuGet measuring time of a call and cancelling if too long).

Note: timeout also happen on very simple/small files that should not timeout and are perfectly working, such as https://api.nuget.org/v3-flatcontainer/xenko.assets/index.json (source: https://github.com/xenko3d/xenko/issues/310)

Unsuccessful attempts to mitigate:

I could find RestoreRequest.MaxDegreeOfConcurrency but I think it only affect number of high level tasks, not the number of parallel downloads.
Also, I couldn't find a way to set a better timeout through API: HttpSourceCachedRequest.RequestTimeout is assigned (indirectly) from HttpSourceRequest.DefaultRequestTimeout which is a static field.

Verbose Logs

Recently got a full NuGet log of someone having the issue: XenkoLauncherLog.txt

Sample Project

Our NuGet package download high-level function is there: https://github.com/xenko3d/xenko/blob/master/sources/assets/Xenko.Core.Packages/NugetStore.cs#L293

rrelyea commented 5 years ago

It seems like your users have been hitting this for a while. What different versions of NuGet are they hitting that with? How are you calling NuGet restore?

xen2 commented 5 years ago

@rrelyea With 4.x and 5.x. We are calling NuGet directly through API (cf first post, link to source). I suspect a nuget install Xenko.GameStudio -Version 3.1.0.1-beta02-0550 might result in the same error but I don't have the issue with my ISP/connection so can't confirm.

xen2 commented 5 years ago

@rrelyea I would like to submit a PR to make HttpSourceRequest.DefaultRequestTimeout customizable through API. Is it OK to remove the readonly modifier, or would you like it to be passed as a parameter in all the places creating HttpSourceRequest or HttpSourceCachedRequest?

mrbaley commented 1 year ago

This can also happen when a ([very slow] corporate) firewall scans the packages and needs more than 100000ms for that.

NuGet / Home