AdguardTeam / CoreLibs

Core Adguard libraries
https://adguard.com/
Apache License 2.0
40 stars 7 forks source link

Remove complicated heuristics for $domain modifier #1875

Open Yuki2718 opened 7 months ago

Yuki2718 commented 7 months ago

I mean

$domain modifier matching target domain:

In some cases the $domain modifier can match not only the referrer domain, but also the target domain. This happens when all the following conditions are met:

  1. The request has the document content type
  2. The rule pattern does not match any particular domains
  3. The rule pattern does not contain regular expressions
  4. The $domain modifier contains only excluded domains, e.g. $domain=~example.org|~example.com

This seems to be the culprit of false positive I fixed.

STR:

  1. Add ://www2.etc-$document,domain=com|net|icu
  2. Set languages for Google search to Japanese (and maybe use Japanese VPN).
  3. Search for ETC利用照会サービス
  4. Click ログイン

login

blocked

IMO the heuristics is no more needed, as we now have $to and it should be decided by filter author in actual context.

ameshkov commented 7 months ago

As I recall there were lots of $cookie rules that were using $domain to point to a target domain?

Yuki2718 commented 7 months ago

As I recall there were lots of $cookie rules that were using $domain to point to a target domain?

They can be converted to $to.

ameshkov commented 7 months ago

But are they converted in other filter lists? Like uBO lists for instance?

Yuki2718 commented 7 months ago

uBO doesn't support $cookie in the first place.

ameshkov commented 7 months ago

Sorry, I may be confusing things now, but I recall that in uBO filters something similar was used for other rule types, maybe it was $removeparam?

Yuki2718 commented 7 months ago

I don't remember, but I can say we uBO team always assume $domain works as $from, meaning it is used only for base domain than target.

ameshkov commented 7 months ago

Got it, thank you! I generally like the idea and this whole heuristic was made for compatibility reasons back in the day.

Yuki2718 commented 3 months ago

https://github.com/uBlockOrigin/uBOL-home/issues/156 Somewhat related and AGMV3 is also affected. Do all the AG products now support $to and can I safely change these $domain rules in Japanese filter for malicious sites to $to? E.g. /\/t_([0-9A-Za-z]{32})\?token=\1/$document,domain=cn|com|net|top.

ameshkov commented 3 months ago

All save for Safari-based: https://github.com/AdguardTeam/SafariConverterLib/issues/60 (but I think it can't be properly implemented there anyway)

Yuki2718 commented 3 months ago

It's a bit problematic if we have to use $domain for Safari and $to for MV3. Fortunately there are not too many $document rules with $domain so maybe we can use hint or !#if.

krystian3w commented 3 months ago

Maybe you're talking about a problem with the referer:

https://github.com/AdguardTeam/CoreLibs/issues/1579 https://github.com/AdguardTeam/CoreLibs/issues/1421

In AdGuard 4.2+ add-on, opening links from Google causes a strict block prompt again for good sites. I haven't tested yet replace domain/from with to= to unlock opening good pages and prompt warning for scam/malware.

ameshkov commented 3 months ago

Safari most likely won't be able to apply $to properly anyway

Alex-302 commented 3 months ago

Related problem with 3p filter https://github.com/AdguardTeam/AdguardFilters/issues/185543#issuecomment-2271599288