Unexpected use of HTTP server in Pants script download flow

[christophermaier]

I was trying to automate some Pants updates and was looking to use this invocation to update the pants script itself:

curl --proto "=https" \
    --tlsv1.3 \
    --location \
    --verbose \
    --output ./pants \
    https://pantsbuild.github.io/setup/pants

However, when I run that, it appears that at some point in the flow, I get redirected to an http site:

HTTP/2 301 
< server: GitHub.com
< content-type: text/html
< permissions-policy: interest-cohort=()
< location: http://v1.pantsbuild.org/setup/pants
< x-github-request-id: 4DDA:0AD7:24AE8:607545:61F8148F
< accept-ranges: bytes
< date: Mon, 31 Jan 2022 16:57:01 GMT
< via: 1.1 varnish
< age: 78
< x-served-by: cache-fty21360-FTY
< x-cache: HIT
< x-cache-hits: 1
< x-timer: S1643648221.348955,VS0,VE1
< vary: Accept-Encoding
< x-fastly-request-id: de662d2ef47f018f97d6677537bf7d7ea081cf13
< content-length: 162
< 
* Ignoring the response-body
{ [162 bytes data]
100   162  100   162    0     0   2793      0 --:--:-- --:--:-- --:--:--  2842
* Connection #0 to host pantsbuild.github.io left intact
* Issue another request to this URL: 'http://v1.pantsbuild.org/setup/pants'
* Protocol "http" not supported or disabled in libcurl
* Closing connection -1

[benjyw]

Not sure if this is related/will fix, but the officially supported URL to download is https://static.pantsbuild.org/setup/pants

[christophermaier]

I tried with the new URL and it behaved in the same manner, unfortunately.

[benjyw]

This is related to configuration of custom GH Pages domains.

We have a slightly complicated situation:

The Pants v1 docs are statically served on GH Pages at https://pantsbuild.github.io/ from a dedicated repo (https://github.com/pantsbuild/pantsbuild.github.io/). But we also have second repo (https://github.com/pantsbuild/setup/) that serves GH Pages at a subpath https://pantsbuild.github.io/setup, just for the setup script. This is because GH Pages have to be served under <ORG>.github.io, you only get one subdomain per org.

The first repo is set up to actually serve from a custom domain, v1.pantsbuild.org, with "Enforce HTTPS" turned on. If you go to a URL under https://pantsbuild.github.io/ GH Pages will redirect to that same URL under https://v1.pantsbuild.org/.

The second repo is affected by the config of the first repo, since it's served as part of the same GH Pages site. However for some reason it doesn't handle HTTPS redirects correctly.

[benjyw]

OK, fiddled with the config and waited a while and I think it works now. Can you verify?

[christophermaier]

Yup, that works for me (both URLs). Thanks very much for digging into this!