Meta-RFE: Verification of HTTPS exclusions

[TPS]

I hate to say it, but increased volume of exclusions recently really sends up a 🚩 for me: Does @AdguardTeam have in place some sort of rigorous, public, ongoing (rechecking) verification of HTTPS exclusions? Recently, there've been lists of dozens @ a time. Even if every 1 of those is currently something that's properly verified (hard task, often, especially in languages & regions 1 may not be familiar), given the amount of marketing churn (e.g., banks changing names via mergers, &c), do y'all have steps in place to reverify?

It's not too hard to expect formerly credible domains to be taken over by advertisers or malware.

[ameshkov]

Frankly, we don't yet reverify the domains. We will need to do it eventually, though.

[TPS]

I believe this kind of whitelisting is more dangerous than blacklisting, since the latter can just break functionality, but former bypasses all some/many safety checks. Maybe set this up such that Spyware/Malware list should override this list?

[ameshkov]

but former bypasses all safety checks

Oh, not really. It forbids AG to decrypt HTTPS for that domain, but we still know the domain name and can block access to it.

[ameshkov]

Ok, some safety checks.…

Yeah, some indeed:)

Maybe set this up such that Spyware/Malware list should override this list?

On Android spyware/malware check is rather simple, it checks domains, not page addresses (this would be too slow for mobile devices).

[jawz101]

Many of my financial institutions are not on the list. And I would imagine any page with a login screen should not be filtered. Nor healthcare sites, any government, military, financial, payment, investing, or legal sites.

If I use any of these services or work in any of these industries an app that filters https is risky. I think it should be a more elegant solution than this. Even EV certs aren't popular these days.

Either a list that is tiny to scan, is trained either automatically or by the user, excludes login pages and their subsequent screens... Something besides this.

Or determine the value of https filtering altogether and not show "you're settings are only 72% enabled" if someone opts not to enable this feature. It's my biggest problem with adguard.

[TPS]

@jawz101 My argument is the other way: Since all this filtering (not just HTTPS) happens locally & is never uploaded anyplace (unless 1 chooses automatic crash reports, which will send some configurable level of logging back to AG only), HTTPS whitelisting might be more dangerous to the user, especially with the quantity of not-currently verified exceptions. If you're that confident, you have the Filter Blocklist only mode, as well.

In the years since I opened this issue, how large is this list now? How often have these entries been reverified since inclusion to represent what they were originally?

[jawz101]

I figure this issue is open to keep people from creating new issues and to see if there is ever a valid point.

To that end:

My bank and credit card are not on there, so the list isn't big enough.

And the filtering integrity not only requires trust in the behavior of the AdGuard application but also any list maintainers you use. What if EasyList decided to slip a fancy rule that redirected an Amazon Login page to a different page? I don't know if that is possible but it feels like a risk to me. Every hour or so the EasyList changes because someone changes it.

[TPS]

@jawz101 It seems the Filter Blocklist only option's perfect for you.

[ameshkov]

That's why we limit what's allowed to third party lists. By default they're allowed only blocking or element hiding.

Anyway, this is a common issue to any content blocker, be it AdGuard with HTTPs filtering, or AG browser extension or uBO that have unlimited access to all pages.

What we do besides providing this HTTPS exclusions list is serving most of the lists from our servers so that we could react promptly to any malice of a third party list author. There is nothing surprising in that btw, it happened before multiple times. The last case was a filter maintainer deciding to block some of the websites of a political party(?) he was not fond of.

[ameshkov]

Well my bank and credit card are not on there, so the list isn't big enough.

If you don't mind, please pull request or let us know what we should add.

[ameshkov]

Regarding the original comment, we made it much easier to disable https filtering on a particular website in our desktop apps, it's done just by ticking it off in AdGuard Assistant menu. It won't be that easy to do with Android, but there are ways. For instance, we could provide a "SharingProvider". You tap "Share" in your browser, choose AdGuard, and then configure filtering for a particular website (including https filtering).

[TPS]

@ameshkov A small thing y'all could replicate from other list UI are the toggle ☑s & wholescale text list editor, which would make these lists as simple as those.

[jawz101]

Well my bank and credit card are not on there, so the list isn't big enough.

If you don't mind, please pull request or let us know what we should add.

I'd rather not divulge the financial institutions I use to strangers or in public. And it doesn't do much for everyone else in the world who has their own sensitive sites. Nor do I know if adding a site also excludes its subdomains. Either way, Shallalist is a start on trying to add more banks and financial institutions. If it was a good list it should be 10,000+- not a couple hundred. I'll just opt to keep https filtering off and be annoyed by the message that I haven't fully configured AdGuard. I'm sorry that I am negative. I will never think it's a great idea to hand over stripping encryption to any 3rd party to a handshake- even if it is local.

[TPS]

@jawz101 Again, why's the Filter Blocklist only option not sufficient for you?

[ameshkov]

Either way, Shallalist is a start on trying to add more banks and financial institutions

Oh, thank you! This is a good suggestion.

I'm sorry that I am negative. I will never think it's a great idea to hand over stripping encryption to any 3rd party to a handshake- even if it is local.

No problem, this is a sensitive matter indeed.

On a side note, would it be more acceptable to you if the implementation was open source and under your full control? We're planning to add a content filtering MITM proxy to AdGuard Home eventually.

[jawz101]

@jawz101 Again, why's the Filter Blocklist only option not sufficient for you?

because it's not the default

[TPS]

@jawz101 Again, why's the Filter Blocklist only option not sufficient for you?

because it's not the default

You can set it for your devices. Why would it make sense to be default for everyone?!

[TPS]

On Android spyware/malware check is rather simple, it checks domains, not page addresses (this would be too slow for mobile devices).

Still true, you think? Or have the intervening years of hardware/software improvements made this now feasible?