[meta] Fully HTTPS Browsing (HTTPS Everywhere)

jerryn70 commented 7 years ago

added by @bbinto

Why/User Benefit/User Problem

User might benefit from faster website loading
Avid privacy user expects Focus to support this technology (aligns with our privacy message), as it aligns with our other browser extensions (support for Firefox Desktop, Fennec)
user is protected when entering personal information such as credit cards, passwords etc.

What / Requirements

Implement HTTPS Everywhere API with GeckoView version
Turned on by default

Acceptance Criteria (how do I know when I’m done?)

Ability to turn off/on HTTPS everywhere in case of site breakage

More info

https://www.eff.org/https-everywhere

Enable HTTPS to the sites that don't have HTTPS connection. So that the connection can be more secure. Example : 'brave browser'

sarimak commented 7 years ago

Does it mean that if I type HTTP into the URL bar or follow a HTTP link, the URL gets rewritten to HTTPS? If it is the case, beware that some captive portal login pages become broken when accessed via HTTPS => still needs to be manually disabled in special cases (at least this is my experience with Fennec and HTTPS Everywhere extension).

pocmo commented 7 years ago

We'd copy the approach of HTTPS Everywhere probably: https://www.eff.org/https-everywhere

avipars commented 7 years ago

How would an unencrypted site work with https? @jerryn70

J0WI commented 7 years ago

At HTTPS Everywhere we are currently in the progress to migrate to WebExtensions. There is also libhttpseverywhere, which uses the same rulesets.

See also https://github.com/EFForg/https-everywhere/issues/6937. AFAIK they only use rulesets with "simple redirects" (http => https, same domain). But our rulesets also contain complex redirects with exclusions and mitigation of misconfigured servers, all based on regex.

bbinto commented 6 years ago

User request

J0WI commented 6 years ago

btw: The rulesets can now also be fetched from https-rulesets.org.

brampitoyo commented 6 years ago

@pocmo If we copy the approach used by HTTPS Everywhere and turn it on as default without having any way to turn it off, would it have any unintended side effect?

If it has side effects, we’ll need a menu item inside Settings. Something along this line:

But my preference, whenever possible, is to avoid having any control unless absolutely necessary.

What do you think?

pocmo commented 6 years ago

But my preference, whenever possible, is to avoid having any control unless absolutely necessary.

Absolutely agree with that! :)

@pocmo If we copy the approach used by HTTPS Everywhere and turn it on as default without having any way to turn it off, would it have any unintended side effect?

Good question. As long as we use the latest state of the rule sets I cannot see much going wrong. However the Firefox add-on has an option to turn it on/off. And they mention this in their FAQ:

What if HTTPS Everywhere breaks some site that I use?

This is occasionally possible because of inconsistent support for HTTPS on sites (e.g., when a site seems to support HTTPS access but makes a few, unpredictable, parts of the site unavailable in HTTPS). If you report the problem to us, we can try to fix it. In the meantime, you can disable the rule affecting that particular site in your own copy of HTTPS Everywhere by clicking on the HTTPS Everywhere toolbar button and unchecking the rule for that site.

Overall I think we should not try to hack this into WebView. Instead we should only focus on GeckoView here. Ideally we could make use of the WebExtension and just load it into GeckoView without writing something ourselves. However we do not have WebExtension support in GeckoView yet - although we want to have that eventually.

bbinto commented 6 years ago

@pocmo are you saying we can't / should not plan on shipping this in 7.0 as it depends on GeckoView + WebExtensions?

It will be a nice advantage that we can highlight when announcing the switch to GeckoView then.

Last conversation I remember is that we won't have WebExtensions for a long time :(

In the short time, could we write it ourselves without a WebExtension? What would be the T-shirt size be? Can we write it on top of GeckoView or does snorp's team implement it?

cpeterso commented 6 years ago

Block unencrypted pages Whenever possible, load the encrypted (HTTPS) version of pages

@brampitoyo I would change this wording to "Prefer encrypted pages". HTTPS Everywhere uses HTTPS when available, but still allows unencrypted HTTP for other sites. "Block unencrypted pages" sounds like unencrypted HTTP sites would be blocked, which I assume is not what we're proposing here. :)

There was a Firefox add-on, HTTP Nowhere, that used to implement HTTP blocking functionality.

J0WI commented 6 years ago

[...] but still allows unencrypted HTTP for other sites. "Block unencrypted pages" sounds like unencrypted HTTP sites would be blocked [...]

Actually there is an option in HTTPS Everywhere to do this ;)

brampitoyo commented 6 years ago

As long as we use the latest state of the rule sets I cannot see much going wrong. However the Firefox add-on has an option to turn it on/off.

@pocmo The toggle existed for a good reason, then. If we want to have it under Settings, should we turn it on by default?

I still worry about adding a toggle that, 99% of the time, is always turned on. If this is the case, why clutter the page with one more option?

I would change this wording to "Prefer encrypted pages". HTTPS Everywhere uses HTTPS when available, but still allows unencrypted HTTP for other sites.

@cpeterso My original thinking was to keep the wording of each option consistent, but I agree that saying “block” is not really appropriate here.

This is the perfect time for Brian to come in with some content expertise.

@BrianNJones, here’s our background information:

We’d like to possibly ship HTTPS everywhere. This is all well and good.

What’s less clear is:

Which category should it be put inside the menu? Under “Privacy & Security” makes sense.
“Privacy & Security” has a lot of sub-categories. Which one makes the most sense for this feature?
- Tracking Protection
- Web Content ← I was thinking to put it under this one
- Cookies and Site Data
- Switching Apps
- Data Choices
How should we call and describe this feature?
- Initially, I was thinking to follow the convention that many other controls have and use the word “block”. So, something along the lines of “Block unencrypted pages”.
- However, @cpeterso pointed out that this is incorrect. What this feature will do is to prefer loading encrypted pages at all times. When impossible, it will revert back to HTTP. He suggested “Prefer encrypted pages”.

pocmo commented 6 years ago

@cpeterso had a good argument on a mail thread:

btw, the benefit of HTTPS Everywhere is small these days because Firefox (and other browsers) ship bundled HSTS lists of sites that have explicitly opted into HTTPS-only (by sending the Strict-Transport-Security HTTP header to Mozilla's HSTS sniffing bot). The HTTPS Everywhere lists are community-maintained and not necessarily with the testing or knowledge of the HTTPS site operators.

I wonder if that makes the cost of implementing this too high for the small benefit we get. It may be worth talking to someone inside Mozilla with deep knowledge about HTTPS and its adoption? I do not know who that would be .. but it's very likely that we have such an expert .. :-)

@pocmo The toggle existed for a good reason, then. If we want to have it under Settings, should we turn it on by default?

Maybe we could have no setting and it would be always enabled except if you turn off the master switch? We probably do not need a global setting but just a way to "unbreak this site"?

pocmo commented 6 years ago

It may be worth talking to someone inside Mozilla with deep knowledge about HTTPS and its adoption? I do not know who that would be .. but it's very likely that we have such an expert .. :-)

I've found @jcjones and @mozkeeler on phonebook and have sent them a mail asking them about their opinion. :)

J0WI commented 6 years ago

However, @cpeterso pointed out that this is incorrect. What this feature will do is to prefer loading encrypted pages at all times. When impossible, it will revert back to HTTP.

That's also not 100% correct. We can't detect whether a site supports HTTPS or not. There are some cases where simple redirects to HTTPS are impossible due certificate mismatches, the server forces HTTP on some sites or major mixed content issues. See e.g. our recently updated ruleset for Orcale. We never revert back to HTTP, but there is a mechanism to stop redirecting attempts if the server answers with redirects back to HTTP for several times. Some of those rules have special attributes and are disabled by default in the extension. But they can be enabled by advanced users (e.g. in the case of mixed content, that can be mitigated by the extension itself in most cases). Some rulesets also contain a rule to force the secure flag for cookies, but this feature might be out of scope here.

@cpeterso had a good argument on a mail thread:

btw, the benefit of HTTPS Everywhere is small these days because Firefox (and other browsers) ship bundled HSTS lists of sites that have explicitly opted into HTTPS-only (by sending the Strict-Transport-Security HTTP header to Mozilla's HSTS sniffing bot). The HTTPS Everywhere lists are community-maintained and not necessarily with the testing or knowledge of the HTTPS site operators.

I wonder if that makes the cost of implementing this too high for the small benefit we get. It may be worth talking to someone inside Mozilla with deep knowledge about HTTPS and its adoption? I do not know who that would be .. but it's very likely that we have such an expert .. :-)

The crux with HSTS is, that the first page load is always insecure (if you not type HTTPS manually). See also our FAQ. Preloaded HSTS solves this, but the list in Firefox contains only about ~40'000 domains, whereas HTTPS Everywhere contains ~150'000 (some of them are wildcards, HTTPS Everywhere does not contain domains, that are already preloaded). Some of the rulesets have a huge impact. E.g. we "preload" all domains from Google, Amazon, Microsoft, Github Pages, JQuery, Cloudflare (including AWS/CDN/S3). Most sites just include those services over HTTP or protocol relative urls. The impact might be slightly lower if Tracking Protection is enabled. We do not collect any telemetry data about this, but there are many rulesets that cover the most ranked domains by Alexa.

mozkeeler commented 6 years ago

Preloaded HSTS is definitely more conservative in Firefox than HTTPS Everywhere - it seems like Focus users would benefit from the additional coverage and e.g. rewrite rules.

bbinto commented 6 years ago

@mozkeeler - thanks for adding your expert advice!

So knowing all of this, what are the next steps?

Discuss with legal/policy (@bbinto)
Eng estimation - T-shirt size? (@pocmo + @boek)

pocmo commented 6 years ago

I'll add it to the next GeckoView+Klar meeting agenda. We'll have to discuss with the GV team what the best approach here is.

bbinto commented 6 years ago

Top 3 Fennec web extension: https://docs.google.com/spreadsheets/d/1drGq4JiIIMBUULAl01g3t8Bh3EKrSJM5N2Bv5RBNmGI/edit#gid=0

J0WI commented 6 years ago

@bbinto could you make this document public? It's not accessible without logging in.

bbinto commented 6 years ago

@J0WI sorry I can't open this document for you, however took a screenshot for context re my comment.

pocmo commented 6 years ago

pct_users is percentage of users with addons (e.g. 40% of users with add-ons use ublock) and not percentag of all users (40% of all users use ublock), right?

bbinto commented 6 years ago

Correct, @pocmo 40% of Fennec release users with add-ons use ublock, and overall ~27% of Fennec users have an add-on installed.

vesta0 commented 6 years ago

@pocmo would you please let me know where we landed with this item in regards to dependency on GV and Android Components?

lobontiumira commented 2 years ago

Hello all! Recently, HTTPS-Only mode has been implemented successfully in all builds, RC, Beta and Nightly. I think this ticket can be now closed.

mozilla-mobile / focus-android