frontend: Category links double-redirect through a non-SSL request

[XhmikosR]

Initially posted here, but since I didn't get a reply I thought I'd report it in a separate issue.

C:\Users\XhmikosR\Desktop>curl -ILl https://shields.io/category/coverage
HTTP/2 301
server: GitHub.com
content-type: text/html
location: http://shields.io/category/coverage/
access-control-allow-origin: *
expires: Thu, 28 Feb 2019 17:03:47 GMT
cache-control: max-age=600
x-github-request-id: A6C0:7DE8:1FA40D:28C77D:5C78121B
accept-ranges: bytes
date: Thu, 28 Feb 2019 16:53:48 GMT
via: 1.1 varnish
age: 0
x-served-by: cache-lcy19243-LCY
x-cache: MISS
x-cache-hits: 0
x-timer: S1551372828.935303,VS0,VE78
vary: Accept-Encoding
x-fastly-request-id: 18cda7dd7a4e49ae4c6ca668e868c4b67b4dda12
content-length: 178

HTTP/1.1 301 Moved Permanently
Server: GitHub.com
Content-Type: text/html
Location: https://shields.io/category/coverage/
X-GitHub-Request-Id: 3884:6DFB:3557EF:44E926:5C78121C
Content-Length: 178
Accept-Ranges: bytes
Date: Thu, 28 Feb 2019 16:53:48 GMT
Via: 1.1 varnish
Age: 0
Connection: keep-alive
X-Served-By: cache-lcy19251-LCY
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1551372828.158699,VS0,VE76
Vary: Accept-Encoding
X-Fastly-Request-ID: 0da254706096c015b87fb989bb2074601c43beb4

HTTP/2 200
server: GitHub.com
content-type: text/html; charset=utf-8
last-modified: Thu, 28 Feb 2019 00:11:22 GMT
etag: "5c77272a-9b87"
access-control-allow-origin: *
expires: Thu, 28 Feb 2019 17:03:48 GMT
cache-control: max-age=600
x-github-request-id: 8094:10BF:1D3DAB:25E798:5C78121C
accept-ranges: bytes
date: Thu, 28 Feb 2019 16:53:48 GMT
via: 1.1 varnish
age: 0
x-served-by: cache-lcy19243-LCY
x-cache: MISS
x-cache-hits: 0
x-timer: S1551372828.341464,VS0,VE75
vary: Accept-Encoding
x-fastly-request-id: 074f5f86ea3943959ada76becd15069da24c88d4
content-length: 39815

This seem to happen for all category links.

[paulmelnikow]

Hmm, this is a strange behavior. The most puzzling part is that it redirects to a non-https address. tc39/ecma262#231 is a report of the same issue.

As curl is producing this behavior we can rule out the application as the cause, since curl isn't running the application at all.

I opened a ticket with GitHub support. If they can't fix it we could either restore the trailing slashes, which I'd rather not do, or look at a different hosting option. For example, we could use Netlify or S3. Alternatively perhaps we could serve these requests off our own servers, placing our own CDN in front.

[paulmelnikow]

If GitHub fixes the issue, the double-redirect will go away and we'll have a single redirect. Specifically, https://shields.io/category/coverage will redirect to https://shields.io/category/coverage/, and then after the page loads, Gatsby will modify the location to again read https://shields.io/category/coverage.

With different hosting, we may be able to avoid the redirect / rewriting but I don't expect GitHub to change it, nor is it really causing a problem.

[XhmikosR]

Yeah, it's not super important, I was just scanning the frontend for 404s etc and came across this, so i thought I'd report it.

Personally I'd just add the domain to the HTST preload list, but that doesn't seem to work either with GitHub.

[paulmelnikow]

I definitely think we should fix that https->http redirect.

Getting HSTS preloaded seems like a nice idea, too. Do you know why it doesn't work with GitHub?

[XhmikosR]

It seems the SSL certificate for the www domain isn't valid. https://hstspreload.org/?domain=shields.io

[paulmelnikow]

Hmm, the site already doesn't work at https://www.shields.io/, so if anything, blocking http://www.shields.io/ is probably useful.

[XhmikosR]

I think you are missing something in your DNS. You need to add a CNAME for www. I use Cloudflare and it works fine for me there.

For example http://www.getbootstrap.com/. It still does redirect back to http at some point but finally resolves to https.

[paulmelnikow]

Not sure about that; www is already a CNAME. www.shields.io works without https.

[XhmikosR]

hmm, then it might a limitation of GitHub pages and custom domains with SSL.

[paulmelnikow]

Response from GitHub: they're aware of this issue, but there is no ETA on a solution. They recommend "always trying to include the trailing slash where an unsafe redirect is not ideal."

Given this issue goes back to 2014, I take it to mean that they're not prioritizing high degrees of compatibility or flexibility on GitHub Pages.

[XhmikosR]

Have you thought switching to Netlify?

[paulmelnikow]

There's some discussion at #3026 about evaluating other CDNs. That's been focused on the badge server, though we could consider the frontend as well.

It also seems possible that it might be possible to host everything on one app, which would make deployment easier, especially in a PaaS environment. The frontend would have to be served from a subfolder and the CDN would need to route requests appropriately based on the path. I don't think Cloudflare supports that, but some other CDNs do.

[XhmikosR]

Is there any CDN that offers this stuff without cookies or inline JS and thus passing the traffic through the origin?

Netlify uses their own CDN, not sure what exactly, but this would be only for the website.

I don't think GitHub is actually investing any time on improving GitHub pages for a long time now.

BTW, I still don't get why can't you have the www subdomain use SSL if you are using Cloudflare. I'm doing the same for multiple sites and works fine.

So, if you can make this work, you could add the HSTS header. This doesn't really fix the issue for new users, though.

[XhmikosR]

So, personally, I'd just go back to Cloudflare. This should fix all issues, and everyone knows it's quite good.

If someone has a serious problem with their cookies, they can disable them. It shouldn't affect all other things just for those few people, IMO.