HTTPS by default on openstreetmap.org

grischard commented 7 years ago

This ticket is intended to be a central place to discuss the possibility of switching openstreetmap.org to be https by default.

This is a subject many people feel passionate about, one way or the other, and not a switch that can simply be flicked on and off. Everyone's concerns, issues on the way, hopes, desires, possible solutions, etc. should be discussed or linked here.

Wikipedia did the switch in 2015, The Guardian in 2016.

This isn't about making all content available over https - that's AFAIK already the case, and users of the HTTPS Everywhere will always use HTTPS when loading the website.

The questions that we should address are:

What advantages would a switch to HTTPS by default bring?

A few so far:

Un-breaking "Show my location" on many browsers without needing any development.
Easing the load on the servers if we can serve all content over HTTP/2 (but that's already enabled for the tile servers, which represent most of the global load)
Better security and privacy for everyone - see the reasons that drove Wikipedia to switch. Make logged in users safer from cookie-stealing attacks.
More reliable Referer headers - https pages don't send them to http pages by default

What issues would a switch to HTTPS by default cause?

People on bad connections might have trouble loading the site.
- @systemed is concerned that HTTPS connections are less reliable over bad mobile links.
- Can anyone show cases where openstreetmap.org is reproducibly slower or less reliable over https?
- Is this something that can be mitigated or improved on the OWG side, for example by enabling http/2 everywhere?
- I couldn't reproduce the issue by loading https://www.httpvshttps.com on a simulated lossy Edge network. (5% packet loss). The tile servers upgrade to http/2 on modern browsers anyway. Unless anyone can show that this effect exists, it sounds a lot like superstition.
People using OSM where HTTPS is blocked couldn't navigate the site.
- petschge (who's on IRC but not on github) was at a private university in Germany in 2015 where a faulty proxy made https browsing impossible. Unfortunately, he's not in contact with them anymore, and can't confirm that the problem still exists. Similarly, @Zverik was in a classroom where HTTPS was blocked in the past. A lot has happened since then: many major sites, e.g. wikipedia and github, now use HSTS, and Wikipedia's 2015 switch might have forced educational institutions to fix this.
- Does anyone know of users who can't access openstreetmap.org over https at all?
- A possible way of measuring this would be by including two identical tracker pixels on openstreetmap.org, one served over http and the other over https, and compare the hit counts after a week.
- Logging in requires https in any case.
Legacy browsers would be locked out.
- SSLLabs says that this would break IE6 on XP and Java 6 users, but that was a tiny proportion of users in November 2016.
- This could also be something that a tracker pixel and a detailed analysis of the user agents in the logs could provide
Content that the iD editor embeds from other sites must be served over https. But that's already the case, since not doing so breaks the embed for users who are already editing over https.

Should we redirect any traffic to HTTPS, or only strongly encourage it?

Forcing and strongly encouraging are different things.

Sending HSTS headers, converting all links to https://, or both, would move most traffic to HTTPS while still replying to legacy HTTP requests. Sending 301 redirects would make sure every client is using https, but make it impossible to use http.

What issues would HSTS cause?

HSTS breaks any site that accesses the OAuth API with http URLs. @mojodna has offered to patch our OAuth library to mitigate the issue.

HSTS could also be selectively enabled, for example only for the http/2 enabled tile servers, which could make the loading of maps faster. Note that clients that are http/2 capable automatically upgrade to https when they connect to a http/2 capable server.

What would technically need to be done to make the switch?

Many pieces of our infrastructure would probably need to be changed or configured differently. How much work would this be for the OWG? Andy is saying it's doable.

Considering the pluses and minuses, is it worth it?

Many of us, myself included, have (strong) opinions on this. I would like us to reach a decision based on measurable facts, and hypothetically review that decision if those facts change over time.

This list of questions is not meant to be exhaustive. I'll try to update the ticket to maintain visibility as comments come in.

don-vip commented 6 years ago

Hello, From https://josm.openstreetmap.de/ticket/10033#comment:38 I discover that all recent browsers now accept http://127.0.0.1:8111 from https (or will very soon). So JOSM is no longer a reason to block https by default on osm.org. I even plan to remove https support on port 8112 as it does not work properly everywhere and appears to be useless now.

grischard commented 6 years ago

What do you all think of already adding HSTS to the tile server?

AFAICT this wouldn't break anything, and modern browsers already upgrade to http2 with tls anyway. We would also hear back from hypothetical users for whom https is completely blocked.

grischard commented 6 years ago

For those of us only following this ticket, see #190 also.

nmxcgeo commented 6 years ago

Hi,

I saw today, that you still send out e-mails with http-links for e.g. comments on your changesets / notes / whatever. As yo are already serving HSTS headers for www.openstreetmap.org I think it's reasonable to change these links to https as well.

Greetings Nmxcgeo

grischard commented 6 years ago

Maybe we could revisit https://github.com/openstreetmap/openstreetmap-website/pull/1341 and https://github.com/openstreetmap/openstreetmap-website/pull/939. What do you think @tomhughes ?

tomhughes commented 6 years ago

I'm not sure we can merge either of those as is, as lots of other people use our code base and they may not be running on an https capable site so we probably need to make it configurable in some way.

I'll have a think about it...

HolgerJeromin commented 6 years ago

At least https://github.com/openstreetmap/openstreetmap-website/pull/939 can be done as Protocol-relative URL

tomhughes commented 6 years ago

Not really - that is code designed to be cut and pasted on a third party site and that would go wrong if it was pasted on an https site and referenced an http only site.

HolgerJeromin commented 6 years ago

Sorry, you are right. Thanks for the explanation.

pnorman commented 5 years ago

We're using https by default now.

genodeftest commented 5 years ago

Thank you very much! PS: Your implementation looks pretty nice! See these good ratings

SSLlabs: A+ https://www.ssllabs.com/ssltest/analyze.html?d=openstreetmap.org&s=130.117.76.7&latest
Securityheaders: https://securityheaders.com/?q=openstreetmap.org&followRedirects=on

grischard commented 5 years ago

Please re-open, tile is indeed the last one that still doesn't redirect.

How to reproduce:

curl -v a.tile.openstreetmap.org/10/11/12.png

Expected:

redirect

Actual result:

200 OK and PNG

The tiles are a hairy yak to shave. The rate limiting stuff we use in squid only works in that version of squid, which doesn't really support modern https and http/2. Upgrading squid breaks things. Re-writing this completely in nginx is possible but would take time, and Andy says that this kind of pre-rendered tile architecture is on its way out in the medium term anyway.

rugk commented 5 years ago

support modern […] http/2

I think this should be enough reason to upgrade it. I mean, especially for multiple connections/tiles like this it should improve speed very much…

grischard commented 5 years ago

Yeah, we have an nginx reverse proxy in front of squid, but only for https.

pnorman commented 3 years ago

Now that we've moved the tile CDN, I think it would be easy to redirect HTTP traffic to HTTPS. This would not impact any users of openstreetmap.org because they would have already browsed to tile.openstreetmap.org and gotten HSTS headers. What it would impact is apps and browsers using a site that has HTTP tiles where the browsers haven't seen HTTPS tiles.

grischard commented 3 years ago

It's even better than that. With HSTS preloading, the only remaining http users are apps, bots, antique browsers more than six years old, and misconfigured reverse proxies.

We could detect user agents that claim to be modern browsers, and block them, since there is no way they would not use https.

Firefishy commented 3 years ago

tile.openstreetmap.org, a.tile.openstreetmap.org, b.tile.openstreetmap.org, c.tile.openstreetmap.org are on the HSTS preloading published list. https://hstspreload.org/?domain=tile.openstreetmap.org

Firefishy commented 3 years ago

We keep HTTP enabled for backward compatibility and to catch apps which are faking web browsers. A HTTP -> HTTPS redirect would be insecure to interception and doubles the number of requests we'd have to process. HSTS preloading is a better route to achieve the same security.

Firefishy commented 3 years ago

Above I conflated this ticket being for HTTPS by default on tile.openstreetmap.org instead of openstreetmap.org. www.openstreetmap.org is already HTTPS default.

rugk commented 3 years ago

Okay, awesome! I agree with all you've said in your last comments, so I think, yes, this issue is solved and I'd thus suggest to close it.

genodeftest commented 3 years ago

Thank you very much!

openstreetmap / operations