Inconsistent handling of HTTPS in package feed

Zhaph commented 1 year ago

Checklist

[X] I have verified this is the correct repository for opening this issue.
[X] I have verified no other issues exist related to my problem.
[X] I have verified this is not an issue for a specific package.
[X] I have verified this issue is not security related.

What You Are Seeing?

Some part of my network appears to be blocking or dropping requests on port 80, such that any requests to http will timeout and not return a response.

The initial request from choco to the Community feed (https://community.chocolatey.org/api/v2/) is made over HTTPS, which succeeds, however all subsequent requests, based on the internal links of the feed are made over HTTP:

For example, when looking for the latest dependency of the notepadplusplus package, the following requests are made:

GET https://community.chocolatey.org/api/v2/FindPackagesById()?id=%27notepadplusplus.install%27 - 200 OK
GET http://community.chocolatey.org/api/v2/FindPackagesById?id='notepadplusplus.install'&$skiptoken='notepadplusplus.install','6.7.2' - Times out

Looking at the feed returned, the key parts are the base property, and the next/previous links, e.g.

<link rel="next" href="http://community.chocolatey.org/api/v2/FindPackagesById?id='notepadplusplus.install'&$skiptoken='notepadplusplus.install','6.7.2'" />

which is directing the client to the http version of the feed.

This means I'm unable to upgrade packages with dependencies (or have to just upgrade the dependencies directly).

What is Expected?

The community feed responds on HTTPS, and directs clients to continue using HTTPS.

How Did You Get This To Happen?

Start Fiddler (or other traffic sniffer/proxy/etc.).

Request an upgrade of a package with dependencies, i.e. choco upgrade notepadplusplus
Observe that the upgrade fails because the dependency can't be found:

choco upgrade notepadplusplus -y
Chocolatey v1.3.1
Upgrading the following packages:
notepadplusplus
By upgrading, you accept licenses for the packages.

You have notepadplusplus v8.5 installed. Version 8.5.2 is available based on your source(s).
[NuGet] [Fiddler] ReadResponse() failed: The server did not return a complete response for this request. Server returned 0 bytes.                                                                                                                                                                                                                                                                                                                                                               
notepadplusplus not upgraded. An error occurred during installation:
 Unable to resolve dependency 'notepadplusplus.install (= 8.5.2)'.
notepadplusplus package files upgrade failed with exit code 1. Performing other installation steps.
The upgrade of notepadplusplus was NOT successful.
notepadplusplus not upgraded. An error occurred during installation:
 Unable to resolve dependency 'notepadplusplus.install (= 8.5.2)'.

Chocolatey upgraded 0/1 packages. 1 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Failures
 - notepadplusplus (exited 1) - notepadplusplus not upgraded. An error occurred during installation:
 Unable to resolve dependency 'notepadplusplus.install (= 8.5.2)'.

Confirm in fiddler that the requests to the community feed are switching back to HTTP.

System Details

Operating System: Windows 10
Windows PowerShell version: PS Core 7.3.4 and Powershell 7
Chocolatey CLI Version: 1.3.1
Chocolatey Licensed Extension version: N/A
Chocolatey License type: N/A
Terminal/Emulator: Terminal and direct Powershell instance.

Installed Packages

7zip 22.1
7zip.install 22.1
awscli 2.10.3
azcopy10 10.17.0
azure-cli 2.47.0
azure-functions-core-tools 4.0.5030
cascadiacode 2111.01
cascadiacodeitalic 2111.01
cascadiacodepl 2111.01
cascadiacodepl-italic 2111.01
cascadiafonts 2111.01
cascadiamonoitalic 2111.01
cascadiamonopl 2111.01
cascadiamonopl-italic 2111.01
chocolatey 1.3.1
chocolatey-compatibility.extension 1.0.0
chocolatey-core.extension 1.4.0
chocolatey-font-helpers.extension 0.0.4
chocolatey-windowsupdate.extension 1.0.5
DotNet4.6.1 4.6.01055.20170308
ffmpeg 6.0
FiraCode 6.2
gimp 2.10.34
git 2.40.0
git.install 2.40.0
gitextensions 4.0.2
jmeter 5.5
KB2919355 1.0.20160915
KB2919442 1.0.20160915
KB2999226 1.0.20181019
KB3033929 1.0.5
KB3035131 1.0.3
KB3118401 1.0.5
keepass 2.53.1
keepass-plugin-keepassnatmsg 2.0.16
keepass.install 2.53.1
MarkdownMonster 2.8.17
mysql.workbench 8.0.31
notepadplusplus 8.5
notepadplusplus.install 8.5
nvm.portable 1.1.9
obs-studio 29.0.2
obs-studio.install 29.0.2
powershell-core 7.3.4
putty 0.78
putty.portable 0.78
rdcman 2.92.1430
sonos-controller 14.20.0
sumatrapdf 3.4.6
sumatrapdf.install 3.4.6.20220629
sysinternals 2023.1.25
vcredist140 14.34.31938
vcredist2015 14.0.24215.20170201
vcredist2017 14.16.27033
vlc 3.0.18
vlc.install 3.0.18
windirstat 1.1.2.20161210
winmerge 2.16.28
yarn 1.22.19
zulu 16.32.15.0

Output Log

https://gist.github.com/Zhaph/a1c866ea67d221cd5ace908a5fe5dfe1

Additional Context

No response

┆Issue is synchronized with this Gitlab issue by Unito

pauby commented 1 year ago

Some part of my network appears to be blocking or dropping requests on port 80, such that any requests to http will timeout and not return a response.

This would appear to be the place you should start troubleshooting as it would appear you have issues on your network. Is there a proxy in play?

Can you reproduce this on another computer inside your network? A computer outside of your network? That would be the first step.

I'm also going to move this to the choco repository.

Zhaph commented 1 year ago

Thanks @pauby, and thanks for the move - wasn't sure if this was considered an issue with the "site" as the issue I was raising was with the contents of the community feed, vs. how choco itself handles the feed - technically asking choco to force everything to secure wouldn't be ideal as I'm sure some people with internal feeds might be happy with HTTP requests rather than applying a certificate to the service. I guess however, as community.chocolatey.org is issuing an STS header (strict-transport-security: max-age=12960000) and issuing a 307 response to force the request to HTTPS (when I connect to it from a completely separate network), it implies that choco's not honouring the first part.

Yep, the issue does seem to be somewhere with my ISP - I get the same behaviour through my mobile network as my broadband, but they're both through the same provider (Vodafone UK), which makes me think that's where the issue fundamentally lies, and connecting through my work VPN seems to work as expected.

However, I still think a better idea would be for the feed to be updated with the correct/preferred scheme.

pauby commented 1 year ago

My mobile is also Vodafone UK. I tethered my laptop to it and have no problems so I don't think the issue is there.

Zhaph commented 1 year ago

Good to know. Looks like it was something within either Vodafone's SecureNet controls or "Adult Content Restrictions", disabling those and everything works now...

pauby commented 1 year ago

@Zhaph Thanks for confirming that.

Now that the Chocolatey CLI issue is resolved, I've spoken to the team and the issue with the Chocolatey Community Repository is something that should be investigated.

I've going to reopen this and move it back to where you opened it originally(as that repository covers that repository).

Zhaph commented 1 year ago

Cheers @pauby, makes sense. Seeing as the server is issuing a 307 redirect to the HTTPS version anyway, it's probably for the best that the feed is using HTTPS internally at least.

chocolatey / home