Github downloads fail on linux (Error getting response stream (ReadDone2): ReceiveFailure)

[shalroth]

The only similar issue I've been able to find isn't a CKAN issue and is likely not helpful here, but I'll link it anyway: https://bugzilla.xamarin.com/show_bug.cgi?id=19673

CKAN Version: CKAN 1.22.3+64309722a446

Operating System: Gentoo Linux Mono JIT compiler version 4.8.0 (Stable 4.8.0.524/9d74414 Sat Jun 17 02:18:11 MDT 2017)

The issue you are experiencing:

Several mods I'm trying to install have installed successfully... but others fail to download. KerbalEngineerRedux is my "case study" mod that I'll use as a specific example here, but I'm getting the same behaviour from several others.

I'm trying to install KerbalEngineerRedux using this command line: $> mono ckan.exe install -d --headless KerbalEngineerRedux

From the debug output, here are the mods I've been able to successfully install:

1335 [1] DEBUG CKAN.SanityChecker (null) - xScience v5.7 provides xScience
1335 [1] DEBUG CKAN.SanityChecker (null) - HoldTheMonoprop 1.01 provides HoldTheMonoprop
1335 [1] DEBUG CKAN.SanityChecker (null) - ModuleManager 2.8.0 provides ModuleManager
1335 [1] DEBUG CKAN.SanityChecker (null) - KIS 1.5.0 provides KIS
1335 [1] DEBUG CKAN.SanityChecker (null) - KAS 0.6.3.0 provides KAS
1335 [1] DEBUG CKAN.SanityChecker (null) - CommunityTechTree 1:3.1.1 provides CommunityTechTree
1335 [1] DEBUG CKAN.SanityChecker (null) - SETI-ProbeParts 1.2.2.1 provides SETI-ProbeParts
1335 [1] DEBUG CKAN.SanityChecker (null) - SurfaceExperimentPack v2.3 provides SurfaceExperimentPack
1335 [1] DEBUG CKAN.SanityChecker (null) - BetterBurnTime 1.5.4 provides BetterBurnTime
1336 [1] DEBUG CKAN.SanityChecker (null) - xScience v5.7 provides xScience
1336 [1] DEBUG CKAN.SanityChecker (null) - HoldTheMonoprop 1.01 provides HoldTheMonoprop
1336 [1] DEBUG CKAN.SanityChecker (null) - ModuleManager 2.8.0 provides ModuleManager
1336 [1] DEBUG CKAN.SanityChecker (null) - KIS 1.5.0 provides KIS
1336 [1] DEBUG CKAN.SanityChecker (null) - KAS 0.6.3.0 provides KAS
1336 [1] DEBUG CKAN.SanityChecker (null) - CommunityTechTree 1:3.1.1 provides CommunityTechTree
1336 [1] DEBUG CKAN.SanityChecker (null) - SETI-ProbeParts 1.2.2.1 provides SETI-ProbeParts
1336 [1] DEBUG CKAN.SanityChecker (null) - SurfaceExperimentPack v2.3 provides SurfaceExperimentPack
1336 [1] DEBUG CKAN.SanityChecker (null) - BetterBurnTime 1.5.4 provides BetterBurnTime

Skipping ahead to the actual install where the failure happens:

About to install...

1953 [1] DEBUG CKAN.NetFileCache (null) - Checking cache for https://github.com/CYBUTEK/KerbalEngineer/releases/download/1.1.3.0/KerbalEngineer-1.1.3.0.zip
1955 [1] DEBUG CKAN.NetFileCache (null) - Rebuilding cache index
 * Kerbal Engineer Redux 1.1.3.0

Downloading "https://github.com/CYBUTEK/KerbalEngineer/releases/download/1.1.3.0/KerbalEngineer-1.1.3.0.zip"
1984 [1] DEBUG CKAN.NetAsyncDownloader (null) - Waiting for downloads to finish...
25989 [Threadpool worker] INFO CKAN.NetAsyncDownloader (null) - Error downloading https://github.com/CYBUTEK/KerbalEngineer/releases/download/1.1.3.0/KerbalEngineer-1.1.3.0.zip: System.Net.WebException: Error getting response stream (ReadDone2): ReceiveFailure ---> System.Exception:   at System.Net.WebConnection.HandleError (System.Net.WebExceptionStatus st, System.Exception e, System.String where) [0x00000] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at System.Net.WebConnection.ReadDone (System.IAsyncResult result) [0x00000] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at System.Net.LazyAsyncResult.Complete (System.IntPtr userToken) [0x00000] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at System.Net.LazyAsyncResult.ProtectedInvokeCallback (System.Object result, System.IntPtr userToken) [0x00000] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at System.Net.LazyAsyncResult.InvokeCallback (System.Object result) [0x00000] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at Mono.Net.Security.AsyncProtocolRequest.StartOperation () [0x00000] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at Mono.Net.Security.AsyncProtocolRequest.<StartOperation>m__0 (System.Object _) [0x00000] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (System.Object state) [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0 
  at System.Net.WebConnection.HandleError (System.Net.WebExceptionStatus st, System.Exception e, System.String where) [0x00044] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
   --- End of inner exception stack trace ---
  at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00064] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at System.Net.WebClient.GetWebResponse (System.Net.WebRequest request, System.IAsyncResult result) [0x00000] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
  at System.Net.WebClient.DownloadBitsResponseCallback (System.IAsyncResult result) [0x00015] in <f8255d9ef0594d18ae2c0d97286b9a80>:0 
25993 [Threadpool worker] INFO CKAN.NetAsyncDownloader (null) - All files finished downloading
25993 [Threadpool worker] DEBUG CKAN.NetAsyncDownloader (null) - Signalling completion via callback
Failed to download "https://github.com/CYBUTEK/KerbalEngineer/releases/download/1.1.3.0/KerbalEngineer-1.1.3.0.zip" - error: Error getting response stream (ReadDone2): ReceiveFailure
25994 [Threadpool worker] DEBUG CKAN.NetAsyncModulesDownloader (null) - Cleaning up /tmp/tmp48ede340.tmp
One or more files failed to download, stopped.
25995 [1] DEBUG CKAN.RegistryManager (null) - Dispose of registry at /emu/steam-linux-library/SteamApps/common/Kerbal Space Program/CKAN

The whole command takes ~26s before it fails; I'm guessing there's a 25s timeout on the download.

However, I can wget the file without a problem, and the whole download takes <2s.

I've been able to reproduce this problem every time I've tried over the last several days, so it doesn't look like any kind of intermittent issue. It's unclear why some github-hosted mods hit this problem but others do not.

As other examples, off the top of my head, some other mods I've had this issue with include ContractParser, ProgressParser, CommunityCategoryKit, CrowdSourcedScience.

[arjensmit]

I have a similar problem. Yesterday i tried to install realism overhaul + carreer and got a handful of "fail to download" errors. Today i tried only realism overhaul and it stopped with a single fail to download error Failed to download "https://www.blizzy.de/toolbar/Toolbar-1.7.13.zip" - error: De onderliggende verbinding is gesloten: Kan geen vertrouwde relatie met het beveiligde SSL/TLS-kanaal maken.

Now that single error is not a github URL. The handful i got yesterday were github though.

Ill try some more stuff later today and be more precise in recording the issues, then edit my post with additional info.

[arjensmit]

Ok today i tried again and made screenshots of the process. After getting 3 failed to downloads, i deselected the corresponding 3 mods and tried again. This time i got a different 3 failed downloads.

Finally i just installed with none of the recommended mods installed and now ill try to add them one by one, by finding out if they will download and what they do.

[shalroth]

@arjensmit Have you look at the basic troubleshooting page? It has some notes that might be relevant to you: https://github.com/KSP-CKAN/CKAN/wiki/Basic-troubleshooting I can't read the language those error messages are in, but at least one of them is a 403 error, which is one of the issues talked about there. Can you download the file if you type the url into a browser? If you can, it's definitely not the same issue I'm reporting here.

[shalroth]

The HD KSP was on crashed, so I've had to reinstall everything fresh. Now, I'm getting the ReadDone2 error for every github download (all spacedock and other sites seem fine). I suspect that my initial successes for some github urls was caused by cached downloads avoiding the download entirely.

Furthermore, I fired up the mono docker container using the latest image (5.0.1.1), and I'm getting exactly the same result (ReadDone2 for every github url), so that should rule out something broken in my particular OS (outside the kernel) or mono install.

[shalroth]

Just noticed this 1.22.3 release note: [core] Switch Linux and MacOS to native C# downloads (#2023 by: politas; reviewed: pjf)

So I tried 1.22.2, which uses libcurl on linux. 1.22.2 works without any problems!

[ayan4m1]

This appears to be a bug in Mono though it's been marked resolved for a while and you're on 4.8.

[netkan-bot]

Hey there! I'm a fun-loving automated bot who's responsible for making sure old support tickets get closed out. As we haven't seen any activity on this ticket for a while, we're hoping the problem has been resolved and I'm closing out the ticket automaically. If I'm doing this in error, please add a comment to this ticket to let us know, and we'll re-open it!