search

DandelionSprout / adfilt

The place where I, DandelionSprout, store my web filter lists for countless topics, including my Nordic adblock list. As simple as that, really.
Other
1.3k stars 143 forks source link

General filter chit-chat #7

Closed THEtomaso closed 5 years ago

THEtomaso commented 5 years ago

Edit by Dandelion Sprout 25th of February 2020: GitHub spectacularly failed to tell me that there was a comment limit of 2,500 comments in issue threads?! So now it seems that discussion has been forcibly moved to https://github.com/DandelionSprout/adfilt/issues/63 instead.

———————————————————————————————————

This thread is a megathread about adblock discussions in general. Here one can request syntax help, reproduction confirmations, info about differences between adblockers, assistance with making new lists, and so on. They'll be answered or considered by the biggest Adfilt contributors, and occasionally by members of the uBlock Origin development team (although in an unofficial fashion). (This header section was last updated on the 24th of April 2019 by DandelionSprout.)

None of the rules for this thread are obligatorily enforced, but are considered customary as of at the time of the last edit:

This thread originally started out as a simple report about removing the blurring from the preview of a premium news article, and is still visible below this line:

———————————————————————————————————

Affected site:

https://www.aftenposten.no/

Example:

https://www.aftenposten.no/karriere/Viktoria-16-betaler-n-million-for-en-karrire-som-ender-nar-hun-er-40-12051b.html

Issue:

Cosmetic filtering blocks the "salesposter" for subscription articles. This results in strangely abrupted page layouts, that gives you the impression that the pages haven't loaded properly, or has some type of error on them. As annoying as these type of "salesposters" may look, it's better not to block them, to avoid confusion!

Problem filter:

Dandelion Sprouts norske filtre for ryddigere nettsider

Problem rule:

aftenposten.no##.widgets.widget-salesposter

--

DandelionSprout commented 5 years ago

At least originally, I felt it was a problem that the sales poster also blurred out the lower lines of the articles previews, and that it would therefore be better for people who didn't intend to subscribe, to remove the blur (which it appears to me that I can't do without also blocking the sales poster).

However, this is a perfectly good issue report and feedback, so I'll look into resolving this issue within the next 15min.

THEtomaso commented 5 years ago

I felt it was a problem that the sales poster also blurred out the lower lines of the articles previews

I think it only blures out the last line partially. The line is still readable too, so it's not an issue.

Thanks for the quick fix! :)

DandelionSprout commented 5 years ago

For me it seemed to give a slight blur to the last two lines, but that's just pedantics of mine. 😄

Closing the issue now that it has been fixed.

DandelionSprout commented 5 years ago

Update:

After an hour of trial and error, I eventually uncovered that the line blur stemmed from aftenposten.no##.widgets-salesposter:before, with :before being some kind of pre-emptive pseudo-child-element that I've had very little prior experiences with.

Having now accomplished this win-win situation, I'll add the new and vastly improved entry to the list sometime today. image

THEtomaso commented 5 years ago

aftenposten.no##.widgets-salesposter:before

Nice catch! :)

THEtomaso commented 5 years ago

Same thing @ digi.no

Example: https://www.digi.no/artikler/442745/

Fix: digi.no##.faded-article-content:after

DandelionSprout commented 5 years ago

Splendidly done of you to have researched it enough to have created a fix entry for it. This'll of course be added to the filter list as soon as possible. 💜

THEtomaso commented 5 years ago

Those type of pseudo-element syntaxes works just fine with uBO. ..but after a recent discussion in the Pale Moon forum, I discovered that ABP-based blockers seems to use a different method (see post 7): https://forum.palemoon.org/viewtopic.php?f=46&t=19682/ Perhaps you're able to make sense of it?

DandelionSprout commented 5 years ago

It doesn't seem to be possible to do it in Adblock Plus at all, from what I can determine.

Adblock Plus' syntax is quite the mess, since I suspect that they're using their goal of encouraging better ads, as a bad excuse to refuse to support new features that originate from other adblockers (e.g. URL wildcards á la google.* in element blocking rules, :has-text, etc.). Coupled with poor documentation (to the point where uBlock Origin GitHub issue threads are more informative about ABP features than ABP's official websites), few examples of such ABP-formatted entries in active use, and that I frankly don't even want to test my lists in Adblock Plus due to its simplified UI, indicates that I think that Adblock Plus is outdated and that I can't recommend its everyday usage in all but the most narrow of cases.

DandelionSprout commented 5 years ago

There is the small theorethical possibility that it's possible to trick :-abp-properties into only detecting :before and :after elements, but I wouldn't place my bets on it. Plus that :-abp-properties isn't even supported in uBO or Nano due to performance concerns (Okay, so advanced adblockers aren't 100% universally covering, either, I must admit), so I can't use Nano's element picker to experiment with it.

THEtomaso commented 5 years ago

Thanks for elaborating.

THEtomaso commented 5 years ago

Fix for dagbladet.no: dagbladet.no##.CTA-body-faded

Example: https://www.dagbladet.no/tema/69884034/

THEtomaso commented 5 years ago

Fix for tu.no: tu.no##.faded-article-content:after

Example: https://www.tu.no/artikler/440961/

THEtomaso commented 5 years ago

OFF TOPIC:

Are you located in Norway? I ask because I'm currently trying to solve a strange filter issue, but no one is able to reproduce it. I've pretty much ruled out UA sniffing and other potentially browser-discriminating schemes at this point, so the most likely explanation seems to be that the problem is location-dependent.

So far, the respective filter authors have refused to include the necessary exceptions in their filters, on the count of them not being able to reproduce the issue from their ends. If this situation remains, the best solution would probably be to include the exceptions in your filter, which is currently the best known Norwegian filter available.

Let me know if you're interested, and I'll post the links to the GitHub discussions here.

DandelionSprout commented 5 years ago

Indeed I am a Norwegian who lives in Norway, and who'd be up for such a challenge.

THEtomaso commented 5 years ago

Great!

Here are the two reference links: https://github.com/ryanbr/fanboy-adblock/issues/512/ https://github.com/AdguardTeam/AdguardFilters/issues/20596/

It's a bit to read up on. :)

DandelionSprout commented 5 years ago

I think I could be seeing the cause of the problem. Are you currently using I Don't Care About Cookies (or anything similar) as one of your filter lists?

THEtomaso commented 5 years ago

No, as you can see from my tests, I made sure to troubleshoot it using ONLY the filters in question!

DandelionSprout commented 5 years ago

Ah, okay.

I can confirm that popsugar.*#@##_evidon_banner works splendidly on my end, but this creates for an interesting situation that I need a few minutes to think on. Don't get too heated up in the meantime, I'm working on it.

THEtomaso commented 5 years ago

So, you can confirm that you get a cookie consent warning that looks like this:

filterissue-popsugar3

And that any one of these three blocking rules alone is enough to block it?:

##.evidon-banner
###_evidon_banner
/evidon-banner.
DandelionSprout commented 5 years ago

Yes, I can confirm that I get a cookie consent that looks like that, and that the latter two entries (except ##.evidon-banner) blocks it.

The thing I'm currently pondering on, is whether it'd be correct conduct or not of me to use my Norwegian list to cover up for the mistakes of mostly unrelated annoyances lists. I need some consideration time on it, and in the meantime there's nothing that stops you from placing popsugar.*#@##_evidon_banner into your personal entries in uBO, so that it's at least fixed on your end.

THEtomaso commented 5 years ago

Yes, I can confirm that I get a cookie consent that looks like that, and that the latter two entries (except ##.evidon-banner) blocks it.

Are you absolutely sure that you tested this properly, and that this rule isn't whitelisted somewhere in your filters? Because from my end, ##.evidon-banner completely blocks it! (Whitelisted cosmetic rules aren't visible in uBO's logger, you know)

The thing I'm currently pondering on, is whether it'd be correct conduct or not of me to use my Norwegian list to cover up for the mistakes of mostly unrelated annoyances lists.

No worries. Given this new "evidence", I think we'll be able to convice the other filter authors to include the necessary exceptions in their own filters.

in the meantime there's nothing that stops you from placing popsugar.*#@##_evidon_banner into your personal entries in uBO, so that it's at least fixed on your end.

Of course, I've already done that. But my goal is to prevent others from running into the same problem. :)

DandelionSprout commented 5 years ago

Are you absolutely sure that you tested this properly, and that this rule isn't whitelisted somewhere in your filters?

Okay, my fault, I tested it again with ##.evidon-banner and it blocks it.

I don't think this is a Norway-only problem: Because GDPR is involved in this, I'm thinking that Popsugar may have chosen (poorly) to differ between EU and non-EU visitors, but I'm trying to look for more proof of that they've done that. I've asked my friends about what they're seeing, one lives in an EU country, and one lives in a non-EU-nor-EEC Eastern European country, and I'm awaiting their replies.

THEtomaso commented 5 years ago

Great, the issue have been confirmed then!! I'll look into convincing the respective filter authors now.. :)

DandelionSprout commented 5 years ago

I wouldn't want to come off as aggressive to them before I've got the facts on the table. The worst that can happen if you nag them enough, is that they'll lock those threads and refuse to help you at all.

DandelionSprout commented 5 years ago

Moreover my Eastern European friend claims to receive the GDRP note, so I need to look deeper into the Javascript files that are involved in this.

THEtomaso commented 5 years ago

I wouldn't want to come off as aggressive to them before I've got the facts on the table.

I've tested this to the point of getting ridicilous, so the three problem rules are a absolute certainty!

The worst that can happen if you nag them enough, is that they'll lock those threads and refuse to help you at all.

No worries, I've reported plenty of issues to these guys, and ryanbr have never struck me as being unreasonable. :)

THEtomaso commented 5 years ago

Moreover my Eastern European friend claims to receive the GDRP note, so I need to look deeper into the Javascript files that are involved in this.

Yeah, but do they get the same one? If you take a look at those issue reports of mine, you'll see that some people (probably in EU countries) get a completely different consent dialog all together!

DandelionSprout commented 5 years ago

I have hereby solved the problem! 🎉

I am 95% certain that this settings.js Javascript file that is referred to in the cookie warning page's source code, determines which cookie notification that users of different countries receive. And there's a lot of different combos of factor that they (questionably) try to account for.

Let's take the string {consentRequired:0,consentid:1,accessid:2,l2enabled:0,iabEnabled:0,gdprEnabled:0,dataRightsType:0,rightslinkId:0,dataRightsFormEmailId:0,privacyLinkId:1,vendor:22},17:. In that string, I am currently of the belief that 17 is the country ID for Norway, as can be seen when a Norwegian user goes to https://c.evidon.com/geo/country.js, and that vendor: is the cookie warning variant that shall be used for that country ID. Vendor:22 is shared with about 90 other locales, apparently.

Likewise, when the country ID is 13 for Russia, they get a vendor: tag of 23, with the outcome for Russian visitors to popsugar.co.uk being said to be the mid-adjusted and large GDPR version. (cont.)

THEtomaso commented 5 years ago

Great work!!

I'm aware that I've been.. well.. "overly persistent" in those reports of mine, but that was just because I knew that I was right! :)

Thanks for looking into this, DandelionSprout!

DandelionSprout commented 5 years ago

So this is the moment where I would begin to push them a little bit. If they don't see the bottom banner version, ask them what number the 'id': tag is when they visit https://c.evidon.com/geo/country.js, and then check up with https://c.evidon.com/sitenotice/4134/popsugar/settings.js whether the country ID's corresponding vendor number in the https://c.evidon.com/sitenotice/4134/popsugar/settings.js lines that starts with {consentRequired:0,consentid:1,accessid:2,l2enabled:0,iabEnabled:0,gdprEnabled:0,dataRightsType:0,rightslinkId:0,dataRightsFormEmailId:0,privacyLinkId:1,vendor: is.

If the vendor number is not 22 and they see the GDPR version (or another non-bottom-banner variant); or their vendor number is 22 and they see the bottom banner version, then my theory is correct.

THEtomaso commented 5 years ago

I've provided them with a link to this discussion, and told them to read from this post and down: https://github.com/DandelionSprout/adfilt/issues/7#issuecomment-410451387 ..so they should be well informed now. :)

THEtomaso commented 5 years ago

BACK ON TOPIC:

Fix for aftenbladet.no: aftenbladet.no##.overlay-wrap:before

Example: https://www.aftenbladet.no/magasin/i/7lG8v4/Fiskeren-og-livsgleden/

:)

DandelionSprout commented 5 years ago

Thanks. I've now added that entry too. Since you've seemed to have quite a lot of suggestions and issues on your mind, I've re-named this thread into what it has become: A place for you to talk about and suggest whichever adblocker-related things with me. 🙂

THEtomaso commented 5 years ago

LOL! Fair enough. :)

THEtomaso commented 5 years ago

I've mailed you a link to my personal filter. Feel free to cherry-pick whichever rules you might find useful. :)

DandelionSprout commented 5 years ago

I don't seem to have received the E-mail in question, unfortunately, assuming that you've sent it within the past 15 minutes.

No, wait, now I've received it.

THEtomaso commented 5 years ago

I sent it right before mentioning it here. Forgot to include a mail subject, but that shouldn't matter.

DandelionSprout commented 5 years ago

I think the lack of a mail subject led to it being flagged as spam and to it being delayed by some minutes, but oh well.

THEtomaso commented 5 years ago

I've re-sent it, with a subject, just in case.

THEtomaso commented 5 years ago

No, wait, now I've received it.

OK, missed that edit. :)

THEtomaso commented 5 years ago

De-blurrer fix for nationen.no: nationen.no##.article-container.locked:after

Example: http://www.nationen.no/landbruk/naeringsforeining-opnar-for-forbod-mot-opsjonsavtalar-pa-jord/

DandelionSprout commented 5 years ago

There were a fair few Norwegian sites in your adblock list that I didn't even know about, so I looked through those sites and added filter entries for them to my list, some of which corresponds with your entries. Thanks again.

DandelionSprout commented 5 years ago

Since your list was concerned about PUP-filled downloads of BurnAware and FileZilla, I suppose that you could be interested in an extension called Redirector, and in the list that I specifically made for that extension, which among other things takes care of cleaning and fixing a fair few poor-quality download links.

THEtomaso commented 5 years ago

Thanks for the tip. I see that I've already got that extension in my ready-to-use folder here though. It's v2.9.3, which was the last Pale Moon compatible version: https://addons.mozilla.org/firefox/addon/redirector/versions/?page=1#version-2.9.3

Think I've pretty much got those type of download issues under control myself, but I feel that it is an ongoing mission of mine to warn others about them. To be honest, I've been trying to cut back on the number of extensions lately, but I've bookmarked your redirector list for testing now, just in case I decide to use that extension again. :)

THEtomaso commented 5 years ago

De-blurrer fix for fvn.no: fvn.no##.overlay-wrap:before

Example: https://www.fvn.no/nyheter/lokalt/i/0EAWW0/Skogbrannen-i-Gjerstad--Fortsatt-godt-med-royk-pa-stedet/

DandelionSprout commented 5 years ago

I've now added the FVN filter as well. 🙂

You don't seem to me to be much of a Chrome fan, but I admit to using that browser for most intents and purposes. After all, in sharp contrast to when Internet Explorer was everyone's go-to browser, Chrome is at least mostly standards-adhering and is frequently updated, and it has quite a lot of extensions to choose from, of which I currently use 18 of. But I too have taken some measures to cut down on extensions: I now use Nano Adblocker for purposes that I had previously used Stylish and Stop Autoplay Next For YouTube for, so I've been able to at least reduce my installed extensions by 2. 😅

I do however also use Basilisk Browser (made by the Pale Moon team) for the specific purpose of playing old web games every few months. I do in fact recommend publicly to those people out there who use Internet Explorer to play Silverlight and Unity Web Player stuff, to switch over to Basilisk, since it supports Silverlight, Unity Web Player, and uBlock Origin. ✅

THEtomaso commented 5 years ago

I take it that you found the blocking rule for Chrome in my filter? :) You are correct, I avoid Chrome like the plague! Its number 1 purpose is to collect user data, because that's what Google is all about!

Basilisk is just Moonchild Productions's test browser. Meaning that they only use it to test features that might eventually end up in Pale Moon. Personally, I can't stand the Australis interface, and I don't want to use bloated extensions (like CTR) to get around it, so using Basilisk is pretty much out of the question for me.

Personally, I highly recommend that you give Pale Moon 28 a try. It's scheduled ro be released sometime this month, but you can already download the latest beta builds here: http://www.palemoon.org/unstable/ This branch markes a huge milestone in Pale Moon's development, and brings the browser up to long-awaited specs, regarding stuff like ES6/7, HTML5, DOM and CSS. I've been trying it for a while now, and I'm very impressed with what their dev. team has accomplished. With Goanna (their custom Gecko engine) running under their new Unified XUL Platform (UXP), the survival of Mozilla's legacy branch is finally ensured! Come September, when Mozilla discontinues their Firefox 52 ESR branch, Pale Moon will be the only FF legacy browser with realistic prospects to survive. The rest of them are doomed!: https://www.computerworld.com/article/3199425/web-browsers/top-web-browsers-2018-ie-edge-and-firefox-return-to-the-road-of-ruin.html

DandelionSprout commented 5 years ago

In contrast, Pale Moon is not my kind of browser. I'm 23 years old, meaning that I was just 10 years old when Firefox's oldest (and Pale Moon's current?) GUI was a thing, so I don't have much of a memory connection to it. Moreover, Pale Moon's lack of support for Web Extensions (as per https://forum.palemoon.org/viewtopic.php?t=15928) is a fatal flaw for me, since most extension producers have discontinued their pre-WebExt XPI versions, which is a flaw that has already slowly begun to cause me problems on Thunderbird.

And, yeah, I did come across that blocking rule of yours. 😅

THEtomaso commented 5 years ago

Well, Pale Moon's complete lack of telemetry (which is quite unique these days) is reason enough to use it, IMO. To quote Moonchild (Pale Moon's lead developer) on the subject:

Working on removing telemetry code from the core actually brought to light just to what excruciating detail Mozilla collects data through their telemetry. It's a real mountain - more data than I can really understand a company like that would ever need to collect from their browser users to improve their product. It looks very much like OCD, to be honest. I can only assume Microsoft is doing the same to a similar level.

Speaking of uBO; Are you using Decentraleyes?: https://decentraleyes.org/ It's an excellent companion to adblockers, as it intercepts requests to big corp. CDNs and delivers the content (which comes bundled with the extension) locally instead! No configuration is required (if Decentraleyes is unable to deliver the content, it will be fetched normally). Even for users that don't care about privacy, this will speed up their browsing experience a tiny bit. :)