search

EFForg / https-everywhere

A browser extension that encrypts your communications with many websites that offer HTTPS but still allow unencrypted connections.
https://eff.org/https-everywhere
Other
3.36k stars 1.09k forks source link

Remove right-wildcard targets (if possible) #11203

Closed cschanaj closed 2 years ago

cschanaj commented 7 years ago

This issue might take a long long long time to complete since a rewrite is not always possible.

Using a modified command from https://github.com/EFForg/https-everywhere/pull/11187#issuecomment-315766782

egrep -l '<target\s+host=\s*"[^"]*\.\*"\s*/>' *.xml
Bisaloo commented 7 years ago

From https://github.com/EFForg/https-everywhere/issues/1327#issuecomment-178265893:

Yeah, I think it would be useful to run a script that does lookups for all the ICANN public suffixes to expand the right-side wildcards to the things they actually resolve to.

It's probably not necessary in many cases because the wildcard actually only covers a handful of suffixes but it may be useful for Google related rulesets for example.

ghost commented 7 years ago

https://publicsuffix.org/

cschanaj commented 7 years ago

I agree that using a script to expand the right wildcard is a probable approach, but popular sites like Google might own nearly all suffix for its domains. It might result in a (number of) huge ruleset with a overwhelming number of target leading to long term maintenance issue.

I will prefer to whitelist a ruleset from the rule-test if it does not cover only a handful number of the suffix.

Remark: Please be noted that the proposed changes rule-test ignore right wildcard target currently.

Bisaloo commented 7 years ago

The idea of removing right-sided wildcards was supported by two of the project developers. We can always ask @Hainish if he shares their views or if the EFF opinion on this has changed.

It might result in a (number of) huge ruleset with a overwhelming number of target leading to long term maintenance issue.

Don't forget that right-sided wildcards are also a hassle to maintain because they don't work with our current automated tools to disable/remove rulesets (https-everywhere-full-fetch-test and hsts-prune) and possibly with future ones as well.

Hainish commented 7 years ago

An ideal solution would be to include the Public Suffix List in the extension itself, and right-wildcard redirects would only be performed if it matches target minus right-wildcard + public suffix. This way we wouldn't have to validate right-wildcards against the list, we'd be assured that they behave as expected. As it stands, the PSL compressed registers at 65k, which is too large of an overhead to add to the extension for this purpose.

As it stands, we highly discourage right-wildcards, but allow them in some cases. I don't think we should remove them completely if they are useful in removing potentially thousands of targets, such as is the case with Google. Judging where they are appropriate and where they can be removed is a call of the ruleset maintainers.

Bisaloo commented 6 years ago

13682 will remove Google.xml and Google.tld_Subdomains.xml