search

DandelionSprout / adfilt

The place where I, DandelionSprout, store my web filter lists for countless topics, including my Nordic adblock list. As simple as that, really.
Other
1.28k stars 141 forks source link

General filter chit-chat #7

Closed THEtomaso closed 5 years ago

THEtomaso commented 5 years ago

Edit by Dandelion Sprout 25th of February 2020: GitHub spectacularly failed to tell me that there was a comment limit of 2,500 comments in issue threads?! So now it seems that discussion has been forcibly moved to https://github.com/DandelionSprout/adfilt/issues/63 instead.

———————————————————————————————————

This thread is a megathread about adblock discussions in general. Here one can request syntax help, reproduction confirmations, info about differences between adblockers, assistance with making new lists, and so on. They'll be answered or considered by the biggest Adfilt contributors, and occasionally by members of the uBlock Origin development team (although in an unofficial fashion). (This header section was last updated on the 24th of April 2019 by DandelionSprout.)

None of the rules for this thread are obligatorily enforced, but are considered customary as of at the time of the last edit:

This thread originally started out as a simple report about removing the blurring from the preview of a premium news article, and is still visible below this line:

———————————————————————————————————

Affected site:

https://www.aftenposten.no/

Example:

https://www.aftenposten.no/karriere/Viktoria-16-betaler-n-million-for-en-karrire-som-ender-nar-hun-er-40-12051b.html

Issue:

Cosmetic filtering blocks the "salesposter" for subscription articles. This results in strangely abrupted page layouts, that gives you the impression that the pages haven't loaded properly, or has some type of error on them. As annoying as these type of "salesposters" may look, it's better not to block them, to avoid confusion!

Problem filter:

Dandelion Sprouts norske filtre for ryddigere nettsider

Problem rule:

aftenposten.no##.widgets.widget-salesposter

--

DandelionSprout commented 5 years ago

I definitely haven't trusted Firefox in recent years. They say they're open-source and the good guys, but they've nevertheless added all sorts of tacked-on features that no one wanted (Firefox Hello comes to mind), so I find it easy to believe that they're heavy on telemetry as well.

I tried out Decentraleyes a few minutes ago, but it didn't seem to detect a whole lot of CDNs on my end, not even on Google Search or on Twitch. I suspect that it could be due to me using Adguard DNS, which is pretty heavy on anti-tracking. Nevertheless I've now volunteered to translate Decentraleyes to Bokmål, especially since it seemed to be a very short project that I could translate in half an hour. So if my volunteer application is accepted in time, you'll very likely get to use Decentraleyes in Norwegian whenever that extension receives its next update.

THEtomaso commented 5 years ago

I find it easy to believe that they're heavy on telemetry as well.

Mozilla are just amateurs when it comes to collecting user data, compared to Google!

I tried out Decentraleyes a few minutes ago, but it didn't seem to detect a whole lot of CDNs on my end

Give it time. I'm sure that it will still pick up a few things here and there.

I've now volunteered to translate Decentraleyes to Bokmål

Everyting on my computer is in English, but good luck with that translation. I see that I'm still listed as the Norwegian translator for qBittorrent, although I stopped translating it quite some time ago. At some point, the devs. stopped letting their translators simply confirm revised lines, completely deleting them instead. This just became too much work.

DandelionSprout commented 5 years ago

Ah, so you're the translator of qBittorrent? In that case I applaud your translation of it, as I could see back when I used it last year that it was great. In contrast, the translations of older versions of e.g. VLC and PeaZip from ≤2016 that were done by other Norwegian """translators""" whose writing skills equalled those of 10-year olds, were complete shit.

In 2016 I began translating and proofreading a whole bunch of PC and Android apps, more than 40 in total, but I remember being enough impressed by the qBittorrent translation that I decided to not try to improve it. 👍

THEtomaso commented 5 years ago

I remember being enough impressed by the qBittorrent translation that I decided to not try to improve it.

If you want to take over, then be my guest. :) It's a lot to catch up on now.

THEtomaso commented 5 years ago

This video should be mandatory to watch, before people were even allowed to use the internet: https://www.youtube.com/watch?v=FHwl6AyL6js It's long, but very informative, and well worth anyone's time. You Google friends makes a visit @ about 2:15:55 :)

DandelionSprout commented 5 years ago

I run on a peculiar sense of logic, basically. I don't really condone telemetry-heavy services, but when the quality of Google's services exceeds those of low-telemetry services by a significant distance, then I find myself using most of Google's products nevertheless.

It's the same kind of logic that allows me to support the Norwegian consumer revolt against palm oil, but also to make exceptions for those few of Knorr and Oreo's products (e.g. Knorr's broccoli soup) that have no good equals among Toro and Sætre's products.

THEtomaso commented 5 years ago

LOL! Well, I must admit that I use Google too, from time to time. Their search engine is just too damn good to completely ignore!

btw; If you really are interested in translating qBittorrent, then contact me @ https://www.transifex.com/sledgehammer999/qbittorrent/ I'll then accept you to the 'Norwegian Bokmål' team, and see about elevating you to the role of "coordinator".

DandelionSprout commented 5 years ago

I'm not interested at the moment, but maybe I could be interested sometime in the autumn. Thanks for the offer.

THEtomaso commented 5 years ago

ryanbr has silently added the necessary exceptions for popsugar.co.uk to his annoyances filters now. Also, I received a mail from the author of I don't care about cookies, who told me that he'll be removing those "Evidon" rules from his filter too, since they only seems to cause problems.

--

EDIT: A fix has also been added to AdGuard's Annoyances filter now. The nagging seems to have worked. :) At least, all this detective work will now make things easier for similar reports in the future.

THEtomaso commented 5 years ago

Reference: https://github.com/uBlockOrigin/uAssets/issues/1495/#issuecomment-410811259

Since I prefer to avoid creating any unnecessary "noice" over @ uAssets, I'll post my thoughts here instead.. :)

The AAK-Cont Filter For uBlock Origin, which is still available here (FINAL BUILD): https://gitlab.com/xuhaiyang1234/AAK-Cont/raw/feaf54c66a7ceb648224688664e6bb74f6d56cef/FINAL_BUILD/aak-cont-list-ubo.txt ..also blocks the embeded videos @ ITavisen.no with the following rule: ||sixscissors.com^$third-party

Despite the fact that this particular filter isn't updated anymore, there are still a lot of people who use it (including myself). So, you might want to consider including this exception in your filter (which I've already added to my own): @@||sixscissors.com^$third-party,domain=itavisen.no

DandelionSprout commented 5 years ago

I was about to suggest that you could've instead switched over from AAK-Cont to Nano Defender Integration, but I now see that the same filter is in the latter list as well.

And seeing as the latter list is actively maintained and is being used by all sorts of adblocker users, I'll add @@||sixscissors.com^$third-party,domain=itavisen.no to the Norwegian list pretty soon.

THEtomaso commented 5 years ago

Nano Defender Integration

Switched

THEtomaso commented 5 years ago

Hmm.. Since Nano Defender Integration is still being updated, shouldn't the issue with ITavisen.no be reported @ jspenguin2017, so that the exception could be added at the source instead? ..or does your experience with those guys tell you that it would be a waste of time?

DandelionSprout commented 5 years ago

In my eyes he's a good guy, but also a strict and no-nonsense guy. If you want to bring the issue up with him, meticulously follow the issue template he's using for the repo in question. It's perfectly possible that he'll want to help you with it, but if he says no, then he gets the final say.

It could potentially be a small problem that you as a Pale Moon user is unable to use the Nano Defender extension that the list is officially meant to be used with, but for such a basic $third-party-filter I don't think it'd become a problem for such an issue report.

THEtomaso commented 5 years ago

I understand. Issue reported: https://github.com/jspenguin2017/uBlockProtector/issues/978/

THEtomaso commented 5 years ago

Sigh This thing with ITavisen.no turned out to be a rather troublesome affair: https://github.com/jspenguin2017/uBlockProtector/issues/978/#issuecomment-410893132 Let's see if all the different filter authors falls in line.. :)

DandelionSprout commented 5 years ago

I use neither of those two lists that you're mentioning in that comment, so at least I'm not affected by it. 😎 Additionally, under normal circumstances that I know of, an exception entry in one list (e.g. in uBlock Protector or in the Norwegian list) will overrule blocking rules in all other lists. So I don't think that it's a life-or-death matter anymore for Adguard to add the exception entry themselves.

THEtomaso commented 5 years ago

exception entry in one list (e.g. in uBlock Protector or in the Norwegian list) will overrule blocking rules in all other lists.

Of course, but you can't rely on one filter to fix the problems in another one. E.g. if there's a problem with a rule in Filter 1, there's no guarantee that the users that run into this problem uses Filter 2, which just happens to contain a fix.

THEtomaso commented 5 years ago

The current video @ itavisen.no plays just fine, even with the following blocking rules in place:

/cdn-cgi/apps/head/*$script,~third-party
||sixscissors.com^$third-party

Things have either been changed server-side, or it may depend on which video they're showcasing. It's probably the former explanation, due to the fact that both the current video, and the previous one, were fetched from the same source, which is ITavisen's YouTube channel. I suggest that you remove the exception rule from your filter, for the time being. ..then we can keep an eye on things, and see what happens.

DandelionSprout commented 5 years ago

While I don't have a 100% grasp of the situation (as I'm not normally a frequent reader of that website), and am going to bed pretty soon, it deserves mention that ITavisen is the only Norwegian website I know of that is actively trying to bypass adblock tools and filters. Therefore I am of the current belief that we'll have to shut out all the anti-adblock measures they take, and to keep them closed so that they can't make use of those measure angles in the future.

THEtomaso commented 5 years ago

Websites not available in the EU because of GDPR: https://data.verifiedjoseph.com/dataset/websites-not-available-eu-gdpr

A nasty business this GDPR stuff! Both because of all the companies that choose not to adhere to it, and because of the way that it has been implemented.

I'm guessing that these sort of blockings can't be bypassed through the use of some fancy script injector rules, but will instead require the use of VPNs or proxies?

DandelionSprout commented 5 years ago

If all those American websites doesn't give a damn about privacy, technology, non-American people, or common sense, I'd say to ignore those sites and to let them wallow in their own puddles.

And you're correct in that I am personally unaware of any ways to use Nano Adblocker or Redirector to bypass this.

THEtomaso commented 5 years ago

Deblurrer for dinepenger.no: dinepenger.no##.article-body:after

Example: https://www.dinepenger.no/overskudd/24412231

THEtomaso commented 5 years ago

Problem filter:

Dandelion Sprouts norske filtre for ryddigere nettsider

Problem rule:

aftenbladet.no,aftenposten.no,bt.no,e24.no,fvn.no,vg.no##[style="cursor: pointer;"]

Issue:

It removes the 'Play' icon for videos @ VG.no. This gives the false impression that the videos are nothing but simple images.

Example:

https://www.vg.no/nyheter/utenriks/i/zLj2Mb/

Screenshot:

filterissue-vg

Note:

The other sites, which the rule is made for, doesn't seem to be affected.

Possible fix:

-vg.no

DandelionSprout commented 5 years ago

Ah, thanks for telling me. That particular filter was something that was suggested to me by someone who took my filterlist user survey, so I didn't really test it before I added it.

THEtomaso commented 5 years ago

Any idea what that person was trying to achieve with this particular rule?

DandelionSprout commented 5 years ago

To be completely honest: No.

All I know was that I tested out the filter by going to VG.no and see if it was detected by Nano Adblocker's logger (which several of the suggested rules weren't, and thus they weren't included in my list). However, the other two entries of his/her that I accepted,

aftenbladet.no,aftenposten.no,bt.no,e24.no,fvn.no,vg.no#?#span:-abp-contains(annonse)
##iframe[src*="annonse"]

, made perfect sense to me, so I thought "What's the worst that could happen?"

THEtomaso commented 5 years ago

Deblurrer for Medier24.no: medier24.no##.faded-article-content:after

Example: https://www.medier24.no/artikler/444820/

Note: The site also needs some cleaning up here and there. I've added some rules to my own filter, but you'll notice these things yourself, simply by browsing around a bit.

--

EDIT1: BTW; I also cleaned up some "leftovers" @ seher.no earlier today, so you might wanna check out that site too.

--

EDIT2: Damn! Like I feared, some of those elements @ seher.no uses dynamic IDs. Of course, this makes them much harder to block! :(

DandelionSprout commented 5 years ago

I must admit that I hardly knew of Medier24 before, and didn't even expect it to have an economically feasibly large readerbase, since it mostly consisted of stories about journalists criticizing other journalists.

Therefore I thank you for drawing my attention to that site. In addition to your suggested entry, I've found and chosen three other entries that I'll add to the list:

medier24.no##.ad-toppbanner-article
medier24.no##.ad-label-vertical
medier24.no##a[href*="adform.net"]
DandelionSprout commented 5 years ago

I've also added two quick entries for SeHer for now. If their element IDs change before long, then I'd need to think of alternate solutions.

THEtomaso commented 5 years ago

I've found and chosen three other entries that I'll add to the list

Yeah, those two first ones are in my filter too.

--

I've also added two quick entries for SeHer for now. If their element IDs change before long, then I'd need to think of alternate solutions.

I've gone through everyting on that site from scratch now, adding a bunch of new cosmetic rules to my filter (probably way too many, as usual) :) Will check back on things in a few days.

THEtomaso commented 5 years ago

Deblurrer for DN.no: dn.no##.vrs-image-block__image-description-overlay

Example: https://www.dn.no/magasinet/2-1-396146

DandelionSprout commented 5 years ago

Thanks again. 🙂 https://github.com/DandelionSprout/adfilt/commit/236d9ff98f8bf18a7aba1f50d0142ddf1e98d0a3#diff-401a09a691b0b77b9198afe4296661b8

THEtomaso commented 5 years ago

I've noticed that you've included some Pinterest rules in you filter.

This brings up a issue that I've been investigating lately.. I've always had Web Notifications disabled in my browser (dom.webnotifications.enabled = false, in Firefox-based browsers). Pinterest is the only place that I'm aware of, where this configuration produces a problem. They seem to specifically search for the presence, and support, for this API in people's browsers. If it isn't detected, then all Pinterest boards will fail to load!

I think this script is the culprit, as it contains some lines refering to "notification": https://s.pinimg.com/webapp/js/pjs-244-71bf7bf228b5899e8fce.js

What do you think? Is there a way to bypass this check, by targeting the above script with some type of advanced injector rule?

DandelionSprout commented 5 years ago

Checking it out for some minutes, makes me inclined to believe that the involved notifications are not web notifications, but is a forced login notification. And to my negative surprise I can't find a way to get around it through filterlist entries.

I always knew that Pinterest were nothing more than petty art thieves, but this is a new low for them.

THEtomaso commented 5 years ago

I've asked gorhill and his crew the same question here: https://github.com/uBlockOrigin/uAssets/issues/3390/

If they can't solve it, then I guess that it really isn't possible. But of course, there's a fair chance that they'll just close my report, for being irrelevant.

THEtomaso commented 5 years ago

Some might find this to be an interesting read-through: https://github.com/uBlockOrigin/uAssets/issues/3389/

It demonstrates another side-effect (besides ordinary breakage) that should be taken into consideration, when creating generic filter rules.

To be filed under "nice to know". :)

THEtomaso commented 5 years ago

I'm having a hard time getting my head around WebExtensions compatible filter syntaxes.

The old ##script:contains() syntax is so easy to use. I can simply make wild guesses, and in a couple of tries I usually get it right!

Here's an example: https://en.wikipedia.org/wiki/Blade_Runner

If you let the mouse pointer hover over the hyperlinks, popups will appear. Personally, I find those things highly annoying, so I've been blocking them with the following rule: wikipedia.org##script:contains(popups)

Now, the question is; how do I achieve the same result, using a WebExtensions compatible filter syntax?

DandelionSprout commented 5 years ago

This one should be easy enough for me to answer, if I understand your question correctly.

• Version compatible with ABP and uBO: wikipedia.org##script:-abp-contains(popups) • uBO-specific version: wikipedia.org##script:has-text(popups)

DandelionSprout commented 5 years ago

I'm not sure if popups refer to a string of text or to an element, however.

If it's the latter, then the equivalents are -abp-has and has respectively.

THEtomaso commented 5 years ago

Thanks for your input, DS. ..but I've been trying all those combinations, and nothing works. This is what's makes this whole thing so damn frustrating! :(

DandelionSprout commented 5 years ago

Ah, in that case I don't really know of any direct replacements for :contains.

But fear not, because I have created two regular entries that on my end solves the problem! 💪 

wikipedia.org##.mwe-popups-is-tall
wikipedia.org##.mwe-popups-is-not-tall
THEtomaso commented 5 years ago

Yeah, that will work of course, but I'm trying to target scripts here.

Another example is those anti-adblock scripts, which now are used at hundreds of celebrity fansites. Someone made me aware of this, and I discovered that no known filter seems to target them.

Here's an example: http://megan-fox.us/

With the legacy syntax, the anti-adblock script at the site above is easely blocked, like this: megan-fox.us##script:contains(adblock)

Of course, in this particular case, @@megan-fox.us$generichide also does the job. ..but how do I target the script itself, using a WebExtensions compatible syntax?

DandelionSprout commented 5 years ago

https://github.com/gorhill/uBlock/wiki/Inline-script-tag-filtering#caveats claims that :contains is supported on WebExtensions browsers as long as the underlying browser version and the uBO version is sufficiently updated:

• Not supported in Firefox's WebExtensions version of uBO when...
 • Firefox 56 and less is used;
 • uBO version 1.14.23b2 and less used.

But if that's not possible to achieve, or that it doesn't work regardless, then I'm unfortunately out of ideas.

THEtomaso commented 5 years ago

The person that complained to me is using my filter in uBO, under Google Chrome. ..and of course, :contains is • Not supported in Chromium-based browser.

Yes, I've tried to talk this person out of using Chrome, but to no use! :)

Anyways, thanks for taking your time, looking into this, DS.

DandelionSprout commented 5 years ago

WebExtensions is a term that is to the best of my knowledge only used in the Mozilla ecosystem (and to a far smaller degree in Edge), and not in the Chromium one. So that mis-terming made me a bit confused, to be honest.

THEtomaso commented 5 years ago

Ah, I see. Sorry 'bout that. I've always been trying to keep Google stuff at an arm's length, so I was unaware of this fact. Makes sense though, that there are still some differences in these APIs, although they seem to be highly compatibe with one another (at least in the Chrome > FF direction).

okiehsch commented 5 years ago

Sorry if I intrude but wikipedia.org##script:has-text(popups) is the wrong syntax.

wikipedia.org##^script:has-text(popups) is the correct one and fixes your example issue - https://en.wikipedia.org/wiki/Blade_Runner - on my end, as does megan-fox.us##^script:has-text(adblock) for your second issue.

I will fix megan-fox.us in uBO-filters.

Edit: https://github.com/uBlockOrigin/uAssets/commit/fdf166fe3d73a309ebc96640a6f1d5c63f4568dc

at hundreds of celebrity fansites

if you can provide a list I will add all of them.

THEtomaso commented 5 years ago

wikipedia.org##script:has-text(popups) is the wrong syntax. wikipedia.org##^script:has-text(popups) is the correct one

Confirmed! :)

if you can provide a list I will add all of them.

OK, I'll see what I can come up with.

DandelionSprout commented 5 years ago

@okiehsch

wikipedia.org##^script:has-text(popups) is the correct one

Aaaaaaah. I was unaware that ##^script was possible to use. I've been making quite a large number of lists for adblockers over the past 18-ish months, and I always learn something new about which syntaxes one can use with them.