[Bug]: Sourceforge Download Site not available

[chezzer64]

What happened?

The sourceforge download site https://nchc.dl.sourceforge.net appears to be unavailable, causing problems when trying to download apps such as WinSCP and KeePass using the URI provided by Get-EvergreenApp. Not sure if this is a temporary problem with the nchc.dl.sourceforge.net site or if sourceforge URI's provided by Evergreen need amending to point at a different sourceforge downloads destination?

Version

2303.789

What PowerShell edition/s are you running Evergreen on?

Windows PowerShell

Which operating system/s are you running Evergreen on?

Windows 10

Have you reviewed the documentation?

• [X] Troubleshooting at: https://stealthpuppy.com/evergreen/troubleshoot/
• [x] Known issues at: https://stealthpuppy.com/evergreen/issues/

Verbose output

Get-EvergreenApp -Name WinSCP | Test-EvergreenApp

Result URI
------ ---
 False https://nchc.dl.sourceforge.net/project/winscp/WinSCP/5.21.8/WinSCP-5.21.8-Setup.exe

Get-EvergreenApp -Name KeePass | Test-EvergreenApp

Result URI
------ ---
 False https://nchc.dl.sourceforge.net/project/keepass/KeePass%202.x/2.53.1/KeePass-2.53.1.msi
 False https://nchc.dl.sourceforge.net/project/keepass/KeePass%202.x/2.53.1/KeePass-2.53.1-Setup.exe

[aaronparker]

Manually downloading KeePass returns ixpeering.dl.sourceforge.net right now. Unfortunately SourceForge doesn't provide a good way to return mirrors. Hopefully this is temporary.

WinSCP looks to have a new download location, so it might be time to update that application.

[chezzer64]

Hi Aaron, appreciate the quick investigation and all the work you do on Evergreen. I would assume all the following Evergreen Apps would be impacted:

Application Endpoints

---

7zip nchc.dl.sourceforge.net gretl nchc.dl.sourceforge.net KDiff3 nchc.dl.sourceforge.net KeePass nchc.dl.sourceforge.net ProjectLibre nchc.dl.sourceforge.net SAGAGIS nchc.dl.sourceforge.net VeraCrypt nchc.dl.sourceforge.net WinSCP nchc.dl.sourceforge.net

Hopefully the nchc mirror will be back online soon.

[aaronparker]

I did just find a way to resolve the downloads using the mirrors, so the result will be dynamic based on your location; however, it will only return the file type defined by the vendor, e.g. exe, so msi, zip etc. would not also be returned.

[aaronparker]

So KeePass would go from:

Version Architecture Type URI
------- ------------ ---- ---
2.53.1  x86          msi  https://nchc.dl.sourceforge.net/project/keepass/KeePass%202.x/2.53.1/KeePass-2.53.1.msi
2.53.1  x86          exe  https://nchc.dl.sourceforge.net/project/keepass/KeePass%202.x/2.53.1/KeePass-2.53.1-Setup.exe

to:

Version Architecture Type URI
------- ------------ ---- ---
2.53.1  x86          exe  http://ixpeering.dl.sourceforge.net/project/keepass/KeePass%202.x/2.53.1/KeePass-2.53.1-Setup.exe

[aaronparker]

I've added a branch to test this result: 082f09031000d2a3521a45c28ac92bc5fb6dbc2d

This works OK, but returns only the file defined in best_release.json that SourceForge uses to define the primary download for a specific project.

[chezzer64]

Would be a shame not to have the other installer types available but probably preferable to have a more reliable download source for the main installer. So personally, I'd be happy with that update - but Deyda will probably need to implement some changes to NeverRed, as he currently uses the msi for KeePass.
Many thanks.

[NickolajA]

This is a bummer. Would it be possible to add 7-zip's own download URLs directly if SourceForge now won't support the different installation types, like MSI?

[aaronparker]

Get-GitHubRepoRelease has a -ReturnVersionOnly parameter - that value is then used in defining a URL to binaries hosted elsewhere, so the SourceForge function could be updated to do the same

[NickolajA]

It seems things have changed a bit since yesterday. I don't have that mu knowledge into how SF functions, but at least the URL returned by Get-EvergreenApp now takes you to a web page that responds:

https://sourceforge.net/projects/sevenzip/files/7-Zip/22.01/7z2201-x64.msi/download?use_mirror=netcologne

However it's not exactly the setup file you get, if you attempt to simply download the "expected" file from the URL.

[aaronparker]

Here's how the release can be resolved without hard coding anything other than the best_release.json - this approach would be ideal:

$BestRelease = Invoke-RestMethod -Uri "https://sourceforge.net/projects/sevenzip/best_release.json"
$httpWebRequest = [System.Net.WebRequest]::Create($BestRelease.platform_releases.windows.url)
$httpWebRequest.MaximumAutomaticRedirections = 3
$httpWebRequest.AllowAutoRedirect = $true
$Resolved = $httpWebRequest.GetResponse()
$Resolved.ResponseUri.AbsoluteUri

The version number is captured from the response in $BestRelease, so like the GitHub function, I can return just the version number from that JSON response, and then hard code the URLs into Evergreen and do a string replace on the version number to get to https://www.7-zip.org/a/7z2201-x64.exe.

Now if only we could get the vendors to stop using the hot steaming pile of shite that is SourceForge...

[aaronparker]

Here's what can be returned from SourceForge by returning only the installer defined in best_release.json:

Version      : 22.01
Architecture : x64
Type         : exe
Size         : 1575742
Md5          : a6a0f7c173094f8dafef996157751ecf
URI          : http://ixpeering.dl.sourceforge.net/project/sevenzip/7-Zip/22.01/7z2201-x64.exe

[aaronparker]

Here's what will be returned after adding the -ReturnVersionOnly parameter to Get-SourceFourceRepoRelease and hardcoding the URLs from the 7zip site:

Version Architecture Type URI
------- ------------ ---- ---
22.01   x64          exe  https://www.7-zip.org/a/7z2201-x64.exe
22.01   x86          exe  https://www.7-zip.org/a/7z2201.exe
22.01   ARM64        exe  https://www.7-zip.org/a/7z2201-arm64.exe

This would then be similar to the other applications that use SourceForge.

[NickolajA]

Is there a way to also include the MSI file in the above output?

[aaronparker]

I rethought through the solution and now get this:

Version      : 22.01
Architecture : x64
Type         : msi
Size         : 1912320
Md5          : 50515f156ae516461e28dd453230d448
URI          : http://ixpeering.dl.sourceforge.net/project/sevenzip/7-Zip/22.01/7z2201-x64.msi

Version      : 22.01
Architecture : x64
Type         : exe
Size         : 1575742
Md5          : a6a0f7c173094f8dafef996157751ecf
URI          : http://ixpeering.dl.sourceforge.net/project/sevenzip/7-Zip/22.01/7z2201-x64.exe

Version      : 22.01
Architecture : ARM64
Type         : exe
Size         : 1607459
Md5          : 8db963c7d47f982df0723ea8c0599a9e
URI          : http://ixpeering.dl.sourceforge.net/project/sevenzip/7-Zip/22.01/7z2201-arm64.exe

Version      : 22.01
Architecture : ARM32
Type         : exe
Size         : 1658438
Md5          : 109aafc5593286a951c1046658d120ba
URI          : http://ixpeering.dl.sourceforge.net/project/sevenzip/7-Zip/22.01/7z2201-arm.exe

Version      : 22.01
Architecture : x86
Type         : exe
Size         : 1290308
Md5          : 734e95cdbe04f53fe7c28eeaaaad7327
URI          : http://ixpeering.dl.sourceforge.net/project/sevenzip/7-Zip/22.01/7z2201.exe

Version      : 22.01
Architecture : x86
Type         : msi
Size         : 1490432
Md5          : b25400614f24ebd0b76421e596ee18c6
URI          : http://ixpeering.dl.sourceforge.net/project/sevenzip/7-Zip/22.01/7z2201.msi

[aaronparker]

KeePass:

Version      : 2.53.1
Architecture : x86
Type         : exe
Size         : 4408888
Md5          : 0f6def7c3a9ea84e45768b7a02d2cabc
URI          : http://ixpeering.dl.sourceforge.net/project/keepass/KeePass%202.x/2.53.1/KeePass-2.53.1-Setup.exe

Version      : 2.53.1
Architecture : x86
Type         : msi
Size         : 3789312
Md5          : 9687c510596ca019c2d7b25e39aeb50d
URI          : http://ixpeering.dl.sourceforge.net/project/keepass/KeePass%202.x/2.53.1/KeePass-2.53.1.msi

[aaronparker]

Change pushed to development branch so that you can test it out if you like 53c7d38338714b886bb6998a2507e6a3848b4e78

I'll update a release in the next 24 hours

[aaronparker]

Fix released in v2304.791

[chezzer64]

New release seems to be working nicely. Think this issue can be closed? Many thanks.

[aaronparker]

Reopening this as the downloads aren't redirecting to the right place; however, this could be a user agent issue.

If you attempt to download from here via a browser: https://stealthpuppy.com/apptracker/, downloads redirect to the SourceForge site. Using Save-EvergreenApp to download binaries seems to work OK.

Could anyone else confirm this behaviour?