dionysius
commented
1 year ago Stumbled on this as well
After investigating https://github.com/ScoopInstaller/Extras/issues/11586 I figured out that we would need to set a referer in the manifest so we are actually downloading the file instead of getting redirected to the website.
Copied from the linked issue:
C:\Users\Dionysius
λ wget.exe https://geeks3d.com/dl/get/720
--2023-07-14 21:15:42-- https://geeks3d.com/dl/get/720
Resolving geeks3d.com (geeks3d.com)... 2001:1600:13:100:f816:3eff:fe9f:d027, 195.15.220.232
Connecting to geeks3d.com (geeks3d.com)|2001:1600:13:100:f816:3eff:fe9f:d027|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://gpumagick.com/downloads/files/2023/abba3e5ee274a0a5a8755e83accae5e2f36671e967ea36d0f445151a0b7cc1fe/FurMark_1.35.0.0_Setup.exe [following]
--2023-07-14 21:15:42-- https://gpumagick.com/downloads/files/2023/abba3e5ee274a0a5a8755e83accae5e2f36671e967ea36d0f445151a0b7cc1fe/FurMark_1.35.0.0_Setup.exe
Resolving gpumagick.com (gpumagick.com)... 2001:1600:4:13:d294:66ff:fe25:ed39, 83.166.138.104
Connecting to gpumagick.com (gpumagick.com)|2001:1600:4:13:d294:66ff:fe25:ed39|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://geeks3d.com/dlz/ [following]
--2023-07-14 21:15:42-- https://geeks3d.com/dlz/
Connecting to geeks3d.com (geeks3d.com)|2001:1600:13:100:f816:3eff:fe9f:d027|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: '720.4'
720.4 [ <=> ] 138.66K --.-KB/s in 0.02s
2023-07-14 21:15:42 (7.51 MB/s) - '720.4' saved [141989]
C:\Users\Dionysius
λ file 720.4
720.4: HTML document, ASCII text, with very long lines (9721), with CRLF, LF line terminatorsvs
C:\Users\Dionysius
λ wget.exe --referer="https://geeks3d.com/dlz/" https://geeks3d.com/dl/get/720
--2023-07-14 21:16:58-- https://geeks3d.com/dl/get/720
Resolving geeks3d.com (geeks3d.com)... 2001:1600:13:100:f816:3eff:fe9f:d027, 195.15.220.232
Connecting to geeks3d.com (geeks3d.com)|2001:1600:13:100:f816:3eff:fe9f:d027|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://gpumagick.com/downloads/files/2023/abba3e5ee274a0a5a8755e83accae5e2f36671e967ea36d0f445151a0b7cc1fe/FurMark_1.35.0.0_Setup.exe [following]
--2023-07-14 21:16:58-- https://gpumagick.com/downloads/files/2023/abba3e5ee274a0a5a8755e83accae5e2f36671e967ea36d0f445151a0b7cc1fe/FurMark_1.35.0.0_Setup.exe
Resolving gpumagick.com (gpumagick.com)... 2001:1600:4:13:d294:66ff:fe25:ed39, 83.166.138.104
Connecting to gpumagick.com (gpumagick.com)|2001:1600:4:13:d294:66ff:fe25:ed39|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14888260 (14M) [application/x-msdos-program]
Saving to: '720.5'
720.5 100%[===================================================================>] 14.20M 67.5MB/s in 0.2s
2023-07-14 21:16:58 (67.5 MB/s) - '720.5' saved [14888260/14888260]
C:\Users\Dionysius
λ file 720.5
720.5: PE32 executable (GUI) Intel 80386, for MS Windows, 10 sections
Feature Request
Is your feature request related to a problem? Please describe.
Some sites block agents, require referers, authorizations, cids, cookies, or similar headers.
Describe the solution you'd like
At the moment, we can set a
checkver.useragentorcheckver.scriptif more control/parsing on the request's required, this is nice, but sometimes it's not enough. The ability to set any header, inautoupdate,autoupdate.hash,homepage, andurlwould be very useful as well, with some global headers prop, and/or more granularly as a sub-prop for each. I'm well aware that there would have to be some work done to translate the set headers to things like aria2, but imo such a feature's well worth it.Describe alternatives you've considered
Some really scuffed
installer.script, with the loss of the benefits of aria2, hash verification, versioning, might work, but that's not something I'd want to do. At that point, a dedicated pwsh script or choco would probably work better.