Change default adblock filter lists

[Retold3202]

Following up from #1237, I'll give my thoughts regarding filter lists we should consider adding or enabling by default:

To start, I think we should keep EasyList & EasyPrivacy enabled by default. These are both very high quality & well respected lists, & great at blocking undesired content. They also make nice use of the ABP syntax.

So first, I think it'd be a great idea to add HaGeZi's Threat Intelligence Feeds. This is a very comprehensive & effective list at blocking malicious domains, and it would help to make up for the lack of Safe Browsing. Similarly, he also has a list for Malicious IPs, which I also think would be worth including.

Some of HaGeZi's other lists could be considered as well, such as his Dynamic DNS blocking & Badware Hoster blocking. I think his Multi ULTIMATE or Multi Pro++ lists could also make nice additions.

Another list I'd like to suggest considering is my BadBlock project, specifically my BadBlock+ list. This is a very carefully curated & high quality list that goes above and beyond most others. Every domain added is carefully researched & considered, as to avoid false positives. I also make nice use of wildcards & the ABP Syntax. Overall, it's very comprehensive, and I'm putting a lot of effort into growing & expanding it as much as possible, and I think it'd make another very nice addition to Cromite's filters.

I think it'd also be worth including the Divested Combined Blocklist. This is a list maintained by Divested Computing Group (behind ex. DivestOS, Mull, Mulch, Hypatia, etc). and includes a variety of high quality sources, and has nice coverage that most other lists seem to miss. They also have a Fingerprinting blocklist that I think would also make a nice addition if its compatible.

Another list that could be considered is the Actually Legitimate URL Shortener Tool. It is very effective at removing tracking parameters from URLs. My only concern would be that I'm unsure if it's compatible with Cromite's ABP or not, would require testing. I think it could also be worth adding Dandelion Sprout's Anti-Malware List, for similar reasons to HaGeZi's Threat Intelligence Feeds.

I think it'd also be worth including EasyList Cookie. This is great at blocking obnoxious cookie banners and improving privacy.

Additionally, while I'm not sure if it should be enabled by default, I think it'd also be worth at least including Yokoffing's 'Block third party fonts' list. This blocks unnecessary fonts used for tracking on webpages, while also unbreaking pages that legitimately need them.

Another list we could consider adding but not enabling by default is 1Hosts Pro. This is another nice and comprehensive list that I personally use, but it does tend to cause breakage from time to time, so that's why I'd be reluctant enabling it by default.

I think that this is a nice starting point for a discussion, and I'd be curious to hear your thoughts as well as what anyone else has to say or suggest.

[uazo]

thank you for the detailed list. I anticipate that it's not my priority right now, also because I don't know the world of adblock lists and so I have to document myself first.

However, I anticipate the goals I would like to set for cromite:

• defining default active lists
• check whether it is possible to check lists automatically

regarding the last point is of fundamental importance, because the blocking made by the lists must not be at the expense of privacy or security: if you deactivate functions or scripts, they become nastier, like those of Google, who first try to display ads trying to preserve privacy and if they fail to do so, they don't give a damn and in order to show them, they exploit mechanisms that lead to sending private cookies from the site.

additionally bypassing adblocks is easy and countermeasure methods ineffective or temporary. I can tell you from experience, I have tried it with #251

give me time to consider how to proceed.

[Retold3202]

check whether it is possible to check lists automatically

💯, I think this would be ideal. I wonder how other content blockers ex. uBlock Origin & AdGuard handle this with their built-in lists? I might try to research this myself. You're right, I think we need some kind of mechanism in place to protect against malicious or questionable rules.

regarding the last point is of fundamental importance, because the blocking made by the lists must not be at the expense of privacy or security

Yes, 100%. I would much rather a more private & secure browser over slightly better ad blocking, I think most would agree.

if you deactivate functions or scripts, they become nastier, like those of Google, who first try to display ads trying to preserve privacy and if they fail to do so, they don't give a damn and in order to show them, they exploit mechanisms that lead to sending private cookies from the site.

Do you have any sources or more info on this? I wouldn't mind investigating this as a list maintainer myself to figure out the best way to handle such cases. What you say doesn't surprise me though.

additionally bypassing adblocks is easy and countermeasure methods ineffective or temporary. I can tell you from experience, I have tried it with https://github.com/uazo/cromite/issues/251

Yeah, content blocking in general is a cat and mouse game :/. I just read through the issue, and I think it might be worth investigating if we could add the uBlock filters as well, since they're especially effective at this. I didn't mention them above though because I'm not sure how compatible they are with ABP, so that will need more investigation.

give me time to consider how to proceed.

👍 Best to do this the right way rather than rushing into it, I agree that we should always put privacy & security first. If there's anything that I can do to help from my end as a list maintainer, please let me know.

[uazo]

Do you have any sources or more info on this?

personally tried, no, I don't think there is anything on the Internet. Just reactivate this patch (from log to actual block) and the ad scripts become bad! obviously those not blocked but only hidden (and there are many).

I just read through the issue, and I think it might be worth investigating if we could add the uBlock filters as well

I will get back to that issue and let you know. from what I had seen, there is a lack of real adblock support from v8, such as blocking not the whole script, but only parts of it. I stopped on the analysis because even if I introduced something about it, it would then be specific to cromite and I would not have time to create specific rules.

If there's anything that I can do to help from my end as a list maintainer, please let me know.

be sure of it!

[wildbeetle]

Oh no, please don't add those blocklists or enable them by default.

I once experienced a case where websites/domains could not be accessed after trying to add Hagezi Threat Intelligence Feeds and others (forgot which ones) to my OpenWRT router. And in my experience, hagezi "Multi LIGHT , Multi NORMAL and Multi PRO" are those that do not cause problems.

You need an admin if you want to use aggressive blocklists and that's not good for end users. You just need “set it and forget it filters” for end user.

I don't have many tried filter lists other than oisd and hagezi. So can't comment on the others.

Also, those lists can simply be handled/managed by app such as rethinkdns, personaldnsfilter, invizible pro and others for system-wide not just the browser.

Honestly, I just want you to add these lists to the default filter so I don't have to add them manually: https://easylist-downloads.adblockplus.org/fanboy-annoyance.txt https://easylist-downloads.adblockplus.org/antiadblockfilters.txt

I've asked for it here https://github.com/uazo/cromite/issues/879#issuecomment-2162937608

[Retold3202]

I once experienced a case where websites/domains could not be accessed after trying to add Hagezi Threat Intelligence Feeds and others (forgot which ones) to my OpenWRT router.

This is not intentional at all and I would highly recommend you report any false positives to Hagezi. I've personally never encountered any false positives from TIF, due to the careful curation & high quality sources. I'm not sure how long ago you used it, so it might be worth trying again, and again, please report any false positives if you experience them.

others (forgot which ones)

My guess would be the others are what caused your false positives, rather than TIF.

You need an admin if you want to use aggressive blocklists and that's not good for end users. You just need “set it and forget it filters” for end user.

I generally agree, I think we should focus on enabling lists known to not cause breakage by default, and maybe we could also include some more aggressive lists, but just not enable them by default & leave to the end user to decide (Like ex. uBlock Origin & AdGuard do for instance). I feel like most of the lists I mentioned are generally safe and shouldn't cause issues, with the main exception of 1Hosts (Pro).

Also, those lists can simply be handled/managed by app such as rethinkdns, personaldnsfilter, invizible pro and others for system-wide not just the browser.

I agree that some of these lists are best used at a DNS level, but we can't count on the majority of average users having DNS protection in place (not to mention also having the specific desired list). Therefore, I think it'd be a good idea to still include some domain lists like this (Other content blockers such as uBlock Origin & AdGuard also include some domain lists), but I agree, we also need more advanced lists as well, such as potentially the uBlock Origin filters if they are compatible.

Honestly, I just want you to add these lists to the default filter so I don't have to add them manually: https://easylist-downloads.adblockplus.org/fanboy-annoyance.txt https://easylist-downloads.adblockplus.org/antiadblockfilters.txt

I agree with Anti-Adblock filters, but I'm a little mixed on Fanboy's Annoyances. I think the problem is "annoyances" are subjective by nature, ex. what one may deem as annoying, another may not. I think it'd be worth taking a closer look at what exactly they define as "annoying" to block, and then deciding from there. I personally always use Fanboy's Annoyances myself and haven't had issues, but again, I think it can be subjective depending on use case & personal preference.

[wildbeetle]

I agree with Anti-Adblock filters, but I'm a little mixed on Fanboy's Annoyances. I think the problem is "annoyances" are subjective by nature, ex. what one may deem as annoying, another may not. I think it'd be worth taking a closer look at what exactly they define as "annoying" to block, and then deciding from there. I personally always use Fanboy's Annoyances myself and haven't had issues, but again, I think it can be subjective depending on use case & personal preference.

As you said add to default doesn't mean enable by default. Moreover, they are supplementary filter lists from easylist that are missing in the default list.

[Retold3202]

As you said add to default doesn't mean enable by default. Moreover, they are supplementary filter lists from easylist that are missing in the default list.

Ah, apologies @wildbeetle I misunderstood your point. I 100% agree that Fanboy's Annoyances should be added by default.

[uazo]

Oh no, please don't add those blocklists or enable them by default.

actually what I want is just to enable them by default, so that cromite appears to behave the same. But before we do that, let's talk about it.