search

DandelionSprout / adfilt

The place where I, DandelionSprout, store my web filter lists for countless topics, including my Nordic adblock list. As simple as that, really.
Other
1.3k stars 143 forks source link

General filter chit-chat #7

Closed THEtomaso closed 5 years ago

THEtomaso commented 5 years ago

Edit by Dandelion Sprout 25th of February 2020: GitHub spectacularly failed to tell me that there was a comment limit of 2,500 comments in issue threads?! So now it seems that discussion has been forcibly moved to https://github.com/DandelionSprout/adfilt/issues/63 instead.

———————————————————————————————————

This thread is a megathread about adblock discussions in general. Here one can request syntax help, reproduction confirmations, info about differences between adblockers, assistance with making new lists, and so on. They'll be answered or considered by the biggest Adfilt contributors, and occasionally by members of the uBlock Origin development team (although in an unofficial fashion). (This header section was last updated on the 24th of April 2019 by DandelionSprout.)

None of the rules for this thread are obligatorily enforced, but are considered customary as of at the time of the last edit:

This thread originally started out as a simple report about removing the blurring from the preview of a premium news article, and is still visible below this line:

———————————————————————————————————

Affected site:

https://www.aftenposten.no/

Example:

https://www.aftenposten.no/karriere/Viktoria-16-betaler-n-million-for-en-karrire-som-ender-nar-hun-er-40-12051b.html

Issue:

Cosmetic filtering blocks the "salesposter" for subscription articles. This results in strangely abrupted page layouts, that gives you the impression that the pages haven't loaded properly, or has some type of error on them. As annoying as these type of "salesposters" may look, it's better not to block them, to avoid confusion!

Problem filter:

Dandelion Sprouts norske filtre for ryddigere nettsider

Problem rule:

aftenposten.no##.widgets.widget-salesposter

--

THEtomaso commented 5 years ago

if you can provide a list I will add all of them.

@okiehsch

Posted here: https://github.com/uBlockOrigin/uAssets/issues/3401/

uBlock-user commented 5 years ago

Aaaaaaah. I was unaware that ##^script was possible to use.

HTML Filtering, only supported in FF 57+

THEtomaso commented 5 years ago

Well, that sucks.. I was trying to find a syntax that will work for all browsers!

okiehsch commented 5 years ago

Well, the old syntax ##script:contains also only worked for Firefox based browsers, Chrome does not support Inline script tag filtering or HTML filtering.

THEtomaso commented 5 years ago

This rule can cause some confusing "issues": ##iframe[src*="annonse"]

E.g. @ Dagbladet.no, which has got some "sponsored articles".

Example (embeded video is blocked): https://www.dagbladet.no/annonse/70179055/ Of course, this being a "sponsored article" and all, you might argue that the rule is simply doing its job. However, I think most people expect the video to be shown.

What makes it confusing is that uBO's logger doesn't even tell you that the rule is blocking anything at all!

DandelionSprout commented 5 years ago

I am not a fan of most types of sponsored articles, but nevertheless I've decided on adding dagbladet.no#@#iframe[src*="annonse"] to the list as an attempted fix, which I'll do within the next few minutes.

THEtomaso commented 5 years ago

I'm not a fan of those type of "articles" either. Simply pointing out the fact that the rule can lead to "mystery" blockings. Any idea as to why uBO's logger makes no mention of it?

DandelionSprout commented 5 years ago

Any idea as to why uBO's logger makes no mention of it?

It did show up in my logger (Nano Adblocker in Chrome), so there could be some minor issues regarding your extension version.

THEtomaso commented 5 years ago

OK. It's actually the first time that I've seen this happen.

THEtomaso commented 5 years ago

Deblurrer for VG.no: vg.no##.v1Os6

Example: https://www.vg.no/forbruker/i/9m49nw/

Note: Looks like a dynamic thing, but it seems to work for all "plus" articles at the moment.

THEtomaso commented 5 years ago

wikipedia.org##script:has-text(popups) is the wrong syntax. wikipedia.org##^script:has-text(popups) is the correct one

Now for the next question.. How do I whitelist those type of rules? Neither #@# nor $badfilter seems to have any effect on them!

DandelionSprout commented 5 years ago

I'm mostly stumpted on this, but I know of one thing that could come very much in handy for situations similar to these, but which is very difficult to learn: Xpath.

The amount of people on this planet who know how to use them well, can be counted on two hands, but they allow for very specific element blocks. Examples that I had to ask for help from others to write:

The most reliable and available person I know of who could've taught you Xpath, would be Frellwit, the guy who makes the Swedish adblock list. But I have enough respect for him that at least I wouldn't want to bother him too much. I also haven't asked him for any lessons myself, seeing as Xpath's syntax looks completely Greek to me.

THEtomaso commented 5 years ago

I know of one thing that could come very much in handy for situations similar to these, but which is very difficult to learn: Xpath.

Thanks again for your insight, DS. Think I'm gonna give this one a pass then, since I really don't know anything about web coding, and mostly rely on a method of hits and misses for my filter rules.

--

The most reliable and available person I know of who could've taught you Xpath, would be Frellwit, the guy who makes the Swedish adblock list.

Yeah, I reported an issue to Frellwit yesterday, and noticed the post where you called him "the best filter author in the world". But I'll follow your example, and avoid bothering him. :)

--

I were messing around the other day, and managed to remove the annoying wallpaper backgrounds at cdon.*, using the following combo:

cdon.*/media-dynamic/images/banner/wallpaper_*.jpg
cdon.*###super-duper-wrapper:style(background-color: rgb(255, 255, 255) !important)

The first rule blocks the images themselves of course, while the second rule (which I found an example for @ uBO's Wiki) replaces the ugly background colors, which sometimes are left in the wallpapers place.

Now, I'm trying to come up with a similar solution for softwareok.com.. The standard filters blocks the most obvious stuff (e.g. adsbygoogle), but we're still left with some nasty red squares, which contains a logo that says "ESPRIT". Example: http://www.softwareok.com/?Download=Q-Dir The squares themselves can't be blocked, since the same elements are used for essential things elsewhere on the site. Therefore, I've blocked the "ESPRIT" image, with the following rule: softwareok.com/img/esprit-logo-*.jpg The only thing that remains is to camouflage the red squares, using the same method that I used for cdon.*. So far I haven't been able to come up with a functioning rule though.

DandelionSprout commented 5 years ago

The only thing that remains is to camouflage the red squares, using the same method that I used for cdon.*. So far I haven't been able to come up with a functioning rule though.

My current best attempt at resolving this, is softwareok.com##table[bgcolor="#FB0200"]

THEtomaso commented 5 years ago

Hey, that works just fine! Thanks! :) Hope I didn't waste too much of your time with this stuff.

DandelionSprout commented 5 years ago

It took me 15min, but thankfully I had some available time tonight to solve such a question. 🙂

THEtomaso commented 5 years ago

BTW; I've been cleaning up some "leftovers" on various Norwegian sites lately. Some of which I weren't aware of until recently. If you scroll through my filter list real quick, you might discover more sites of interest for your Norwegian Filter.

uBlock-user commented 5 years ago

How do I whitelist those type of rules?

It's a TODO - https://github.com/uBlockOrigin/uBlock-issues/issues/6

THEtomaso commented 5 years ago

Ah, I see. I'm using uBO Legacy though, so I might not be able to take advantage of those future changes.

okiehsch commented 5 years ago

I'm mostly stumped on this, but I know of one thing that could come very much in handy for situations similar to these, but which is very difficult to learn: Xpath.

https://devhints.io/xpath for an overview of the syntax.

DandelionSprout commented 5 years ago

@okiehsch Ah, thanks for the link, which I've now placed in my wiki-esque toolbox. I'll see if I'll have time this week to practice on it. —————————————————————— @THEtomaso I looked through your list again and found 6 sites where I was able to find things to filter away. So I'll have to thank you for how you know of more Norwegian websites than I do. Oh, and also:

I saw there were two newssites there that was of a peculiar character: frieord.no and nyhetsspeilet.no. Ordinary newssites are bad enough as it is, but e.g. Frie Ord was essentially the incoherent ramblings of xenophobes who had been too much on 4chan and who had been drinking some very strong coffee. Therefore I can't bring myself to browse through those sites willingly. But I see that you are able to, so therefore I hereby extend a carté blanché to you (THEtomaso) to find adboxes and empty boxes (Not newsletter links or social buttons, for the time being) on those two sites that I can block, and which I'll then add to my list virtually uncritically.

THEtomaso commented 5 years ago

Well, not all of the sites in my filter are places that I actually visit myself. Some I've just visited by accident, while others are sites that users of my filter have specifically asked me to clean up for them.

Nyhetsspeilet.no is indeed a speculative site, and much of what's written there have little hold in reality.

FrieOrd.no on the other hand, is only reporting documented cases, and their reporters never deliberately post any "fake news". They have a clear agenda of course, so you should make up your own conclusions for the articles that you read there. However, some of the their articles can be very informative, since they cover cases that you'll never hear about in mainstream media. Because, let's face it.. Norwegian media is heavily dominated by socialists.

I can respect that you disagree with such sites politically. But all together refusing to browse them.. isn't that taking it a bit far? :)

Anyway, I can look through them for you if you want me to. Just one question; which filters do you want me to have activate while browsing? I assume that you consider your own filter to be a supplement to EasyList.. So, 'EasyList' + your 'Norwegian Filter' then?

DandelionSprout commented 5 years ago

Thanks for the detailed and fair explanation, but I must admit that whenever I see newssites that focus chiefly on immigration skepticism, I have a "Breitbart / British tabloid sense" in my brain (á la Spider-Man's spider sense) that begins to ring. I'm not completely scaredy, however, as I have been able to browse Resett on a few occasions, IIRC.

I also suppose it deserves mention that I'm among those people who have gained the most from Norwegian social democracy. I was diagnosed with autism at age 2 (Upgraded to Aspergers at age 14), and I can thank Norway's social welfare for cheap medical appointments, discounted prices on e.g. Citalopram, disability payments that I can subsist on and then some, and discounted housing loans to own my own apartment. Had I been born in the United States, I'd either be dead or in prison by now. All in all, I have considerable respect for social democracy, and also think that the term "socialist" is being used all willy-nilly in the modern political climate while most users of it don't know what it means.

EasyList + Adblock Warning Removal List + Norwegian Filters is a good starting point, since a considerable minority among my userbase (10~15%) use ABP.

DandelionSprout commented 5 years ago

I'm also a believer of the https://en.wikipedia.org/wiki/Mean_world_syndrome theory, so I usually don't even browse through mainstream newssites unless necessary.

The last thing I'd want to do now, would be to create any sort of political tensions between the two of us, especially as we're working towards a common goal of making an adblock list. But if this carté blanché becomes a success, I seem to have become aware of ~6 other sites that'd be worth looking into: ymte.no, norgesavisen.no, Rapport-X, nattnytt.com, nyhetsblikk.com, and arguably ektenyheter.no.

THEtomaso commented 5 years ago

Sorry to hear about you having Aspergers, DS. ..but also, it's good to hear that you got the welfare that you needed to get by with it. Lets just hope that our small country will be able to provide equally as good for its citizens in the future.

I agree not to turn this conversation into a political debate. Believe me when I say that this is the last thing that I'd want to do!

'EasyList' + 'Adblock Warning Removal List' + 'Norwegian Filters' it is.. I'll check out those sites for you! :)

DandelionSprout commented 5 years ago

Sorry to hear about you having Aspergers, DS.

Thanks for the condolences. It's not quite as bad to live with as it may sound like, but it has a handful of severe drawbacks, as it leaves me susceptible to rage fits during stress, feelings of alienation compared to other humans, getting easily exhausted, and struggling to figure out how to not step on others' feelings. Those drawbacks combined have made me unable to e.g. perform 9-5 jobs and the likes.

The one major advantage in my case, is that it's given me a major knack for sorting and setting up things, which in the past 2-3 years has seen me become able to make adblock lists in droves (The current count is ~28, possibly a world record), translate ~60 programs and apps to Norwegian, building and turning my PC into a considerable workstation, and making all sorts of dinner meals (Many of them self-invented).

THEtomaso commented 5 years ago

Cool! I've read stories before about people being able to learn how to use it to their advantage, like you describe.

Also, I belive reading at some point that Magnus Carlsen have Aspergers. ..or is that just some peoples speculation?

DandelionSprout commented 5 years ago

I've seen people theorise a lot about which people that could have Aspergers. At one point in 2015-ish, people were even discussing if Vladimir Putin had Aspergers, which I can 100% confirm that he doesn't.

In the case of Carlsen, I personally think it's just that he grew up with competitive chess from a very young age, so that he probably didn't interact too much with non-chess children and thus had limited opportunities to socialise with others who weren't chess players or journalists. As such, I don't think he has Aspergers, and I'm pretty sure that he hasn't been diagnosed with it.

THEtomaso commented 5 years ago

Not too much going on at those sites that you listed, in regards to ads slipping through.

Found these:

alternativ-media.com##.wpa
ektenyheter.no##.cm-message
resett.no##.tdm-mobile-full
resett.no##a[href="https://resett.no/annonseweb/"]
resett.no###cookie

1 = Ad box (empty) 2 = Cookie consent 3 = Ad box 4 = Ad box 5 = Cookie consent

Had to bite myself not to block any social media links, back-to-top arrows, next article pop-ups, and sharing buttons. Removing that type of crap has almost become second nature to me at this point! :)

--

Notes:

rapport-x.com redirects to alternativ-media.com

These sites are down:

nattnytt.com
nyhetsblikk.com
ymte.no
DandelionSprout commented 5 years ago

There are several lists out there that are already dedicated to removing social buttons and sharing buttons, such as Fanboy's Social Blocking List, which is among the everyday lists that I use. So that's why I felt it'd be superfluous to add such entries to the Norwegian list as well.

Thanks for the notes as well. I was relying on two articles from 2017 to find sites for you, so I suppose that it's perfectly possible that those articles were a bit outdated. I'll add the 5 entries you found to my list as soon as possible.

THEtomaso commented 5 years ago

There are several lists out there that are already dedicated to removing social buttons and sharing buttons, such as Fanboy's Social Blocking List

Yeah, I'm using several of them (including Fanboy's Social Blocking List). Things are still slipping through though..

DandelionSprout commented 5 years ago

One possibility would've been if you began to maintain your own topical lists (e.g. social button blocking lists) for public use, to fill in the gaps between the existing lists.

I'd be more than happy to add such public-usage lists to FilterLists.com, [once/if] they're made. 🙂

THEtomaso commented 5 years ago

Don't think that I have the patience to seperate those rules from my main list at this point. You're more than welcome to take a look at it yourself, if you want to. But you should be aware that I also block third party logins, wherever I might find them.. Those rules will be difficult (if not impossible in some cases) to distinguish from the other social rules.

DandelionSprout commented 5 years ago

Ah, okay. I didn't want to force you to do it, it was simply meant for if you wanted to resolve your problems with existing social button lists, seeing as it's not the easiest thing ever to submit contributions to Fanboy's lists that I'm personally aware of. 😅

THEtomaso commented 5 years ago

Almost missed this one (ad box): norgesavisen.no##.td-fix-index

THEtomaso commented 5 years ago

Issues reported for EasyList can remain unfixed for a very long time. I guess you should consider adding important EasyList fixes for Norwegian sites to your filter, at least temporarily. Here's an example: https://github.com/easylist/easylist/issues/1996/

DandelionSprout commented 5 years ago

I will need to think a bit through whether it'd be feasible for me to include fixes that have also been suggested for EasyList.

It hinges on two things. 1) EasyList's [willingness/ability] to (not) let suggestions stay in limbo; 2) Whether it's feasible for me to check up on EasyList every now and then to see if the fixes have been applied to them yet.

To determine №1, I'd have to look through their forums and GitHub reports more closely than what I can do this morning. For №2 I have a possible solution, which is for me to paste one list above the other in Sublime Text and then sort by "Unique" lines, but it'd mean I'd have to spend 5-10min on it once every 1-2 weeks, which is time that I cannot 100% guarantee that I'll have.

THEtomaso commented 5 years ago

At least for the issues that I report myself (like the one in my previous post), I can notify you when it has been fixed at the source.

DandelionSprout commented 5 years ago

Fair enough for me. Having now been able to think about it a bit more, I've chosen to add that filter to my list. 🙂

THEtomaso commented 5 years ago

Hey.. Looking at your filter, I now see that you already had the following exception in it: @@||imasdk.googleapis.com/js/sdkloader/ima3.js$script,domain=tv2.no Guess you must have added it at some point within the last 12 days, after I filed my EasyList report?

DandelionSprout commented 5 years ago

Ah, that's right. I received a tip about it in https://github.com/uBlockOrigin/uAssets/commit/bce93548ba9987523191ffe024f9b20f65a27915. At the time I wasn't sure if okiehsch meant for me to add that entry to my list as well or not, so I just as well did so.

THEtomaso commented 5 years ago

Here's another one, which I posted almost a month ago: https://github.com/easylist/easylist/issues/1861/

DandelionSprout commented 5 years ago

I struggle to reproduce the Seher issue on my end in a hurry. Is it still a problem on your end?

DandelionSprout commented 5 years ago

On my end, Seher have changed their image rendering elements from "prebid" to "lazyload", the latter of which is not covered by any mainstream lists. Can you confirm this change on your end too?

THEtomaso commented 5 years ago

Like I wrote in my report, it seems to be a "periodic" issue. Sometimes nothing seems to be affected, while other times, the images that I mentioned gets completely blocked! On the occasions when I've encountered this particular problem, I've tested it quite thoroughly.

THEtomaso commented 5 years ago

Seher have changed their image rendering elements from "prebid" to "lazyload"

I can only speculate that they switch between the two of them.

DandelionSprout commented 5 years ago

Okay, so it's still a problem then. I suppose I can add that entry, then. Give me 2min.

THEtomaso commented 5 years ago

I have a whole bunch of other fixes in my filter list, none of which have been fixed at the sources (some filter authors even refuse to). However, those fixes are for filters which are not as widely used as EasyList. If you're interested, you can look through my filter, and search for "Unbreak" listings. All my "unbreak" rules include notes, pointing out which filter(s) they're for, and a rough description of what they fix. I'll leave it up to you to decide if any of the fixes are worth including in your Norwegian list. :)

DandelionSprout commented 5 years ago

I have yet to test my list with Hosts files, so I suppose I can look into a few of those unbreaks.

I have my eyes on 3-ish entries in the unbreak section, that I could check out sometime tonight or tomorrow whether they still need to be unbroken when using the lists in question.

okiehsch commented 5 years ago

I wasn't sure if okiehsch meant for me to add that entry to my list as well or not, so I just as well did so.

Sorry if my comment looked cryptic to you. uAssets will add filters for site breakage issues. The preferred place for such filters is the corresponding regional list, in my opinion. That is why I informed you about it and now that you have added the filter to your list it can be removed from uBO-unbreak, which I will do later today.