Extend ad-block filtering syntax for adjusting default shield settings

[bbondy]

This task is a proposal to add an extension to the ad block filtering syntax for new rules only we support which would allow changing default shield settings for filters.

The rules that we define would go in a new list in https://github.com/brave/adblock-lists

Some examples would be:

• Disable fingerprint protection on some specific URL fragment or host
• Disable 3rd party cookies on some specific URL fragment or host
• Disable ad-block library on some specific URL fragment or host
• Disable tracking-protection on some specific URL fragment or host
• Disable HTTPS everywhere on some specific URL fragment or host

This would allow us to run Brave in a mode of be as secure + private as possible without breaking sites. It would allow us to fix webcompat issues out of line without doing Brave browser upgrades.

We could add an opt out switch which would allow the user to say never to do default exceptions for web compat reasons.

This would enable us to have a team of people that simply need to write filters to debug and fix web-compat issues and don't need to know how to code the browser. We also would detach web-compat fixing from the normal browser update process. A broken site from our shields could instantly be working without a 2-week release cycle.

Example filtering syntax: Allow all cookies on docs.google.com: ||docs.google.com^$cookies=all

Disable ad-block and tracking protection on a site: ||somedomain.com/some-top-level-url$ad-block,tracking-protection

This feature would also allow us to eventually change our default for fingerprint protection to be on, because we could add known breaking fingerprint protection sites as rules.

One possible consideration is that there is some redundancy for certain shield types. We could leave out things that already have lists. For example we could leave out https everywhere, ad-block, and tracking protection. But include cookies, fingerprint protection, and maybe even script blocking. The main reason for the redundancy though would be that we could save a site from a library bug by disabling the shield for that URL.

Other tasks:

• [ ] Add something to welcome screen to explain this.

/CC for feedback @diracdeltas @lukemulks @davidtemkin @SergeyZhukovsky @jhreis

[lukemulks]

I like this approach. It is flexible enough to train contributors, allows for better FP and the option to opt-out.

I think with any approach like or similar to this, it probably would be helpful to have some tooltips and/or a walkthrough on first install and launch for new users. Probably would not hurt for users that update when this lands to get the same walkthrough. Something very basic, as you've outlined above.

What would also be nice would be a cleaner path for users to test custom filters, and have this syntax extension be something we can test.

Right now, I have to either dig into deep preference settings, or remember and enter about:adblock to land on regional lists and the custom filter field.

Here's what would be really great, which may not bode well for UI people, but bear with me.

In the site shields (lion menu):

We include 2 hyperlinks in the shields settings menu:

1. "regional lists" (anchor URL to land on regional lists in about:brave)

2. "Add custom filters" (anchor url to lead user right to the custom filter field in about:brave

I know there are issues with how many items we have in the existing shields menu, but I suspect we could kill some and add these and still have some refinement.

This helps bridge the flow which currently has a wide gap between our blocking UI and the enhanced and additional options, both of which are a high selling point.

Additionally, if the custom field could be modified to allow for scripts and other snippets, we would essentially be baking in greasemonkey and a really nice custom filter field in one. I think this would be a big win and acq point for adblocking users that are getting less and less confident in existing blocking solutions.

The last thing I will suggest is that we not only allow for custom filters, but expand that filter by including a button that allows the user to save a custom filter ruleset as a custom filter list (could have them added below regional blacklists, and include toggle switches).

If we want to win in the ad blocking space, we want to allow for as much customization with custom filters as possible. Maybe we could also allow users to send their custom lists to us to potentially add or submit to github. Just a thought.

Allowing the save custom filters as a new filter list option would be potentially huge imo, especially if we add the ability to include custom scripts and code snippets.

I am watching other ad blockers have to make secondary extensions for particularly stubborn use cases, and this would allow for something better in the browser that the user could save and sync across devices.

I realize I am off track from the original post, but webcompat would naturally be an area that would fit with these customizations.

It would help crowdsource solutions on the fly from contributors that are already going to chew on these things independently. Having points at which they may be able to limit overhead would likely help with getting them to switch and be active in github.

Long story short, this sounds good to me. Of course we will want to test but this is a clear win to get additional open source engagement, and I would be happy to put a training doc together to help contributors learn the syntax and get running.

[davidtemkin]

@bbondy Good approach. We'll need this no matter how it's exposed to users and no matter what else might be added per @lukemulks. We can design the UX separately, and the opt-in/opt-out will be an interesting discussion.

Will the list be packaged in the browser, or downloaded separately at regular intervals? Sounds like you're describing the latter.

[bbondy]

@lukemulks Agreed on adding about 1 time setup, I think it'd be enough to list this as something on the welcome screen.

For better tools to test the blocking, I agree that needs a lot more work. It's listed as one of the things in the adblock initiative.

Also agree on export / save custom filters future work as well, it wouldn't be part of this task but it's a consideration for sure. It also ties into the right click this is an ad feature.

Custom scripts will be separate too.

But as mentioned we have an initiative an steady work going into this now.

@davidtemkin

Will the list be packaged in the browser, or downloaded separately at regular intervals? Sounds like you're describing the latter.

Lists are and will remain separate download that can be updated out of line. The above task is more about defining new rule types that can control default shield settings, the rules themselves will continue to be distributed in the same way. Although today we can't customize default shield settings because it's not part of ad-block so we have to have hard coded new releases for each of those fixes.

[BrendanEich]

I want to repeat something from Slack: we should be big in our thinking for competitive reasons and to avoid building up too many dropped-shield exceptions.

• Block Scripts sometimes wins and we should consider that on the menu of site hacks.
• ITP 2.0 has stuff we may want which could avoid just dropping shields, @jumde and @snyderp are evaluating.
• AdGraph (sorry, may not get all colleagues but @snyderp and @UmarIqbal know them) may lead to other site hacks. not looking to scope-creep this issue, but wanted to mention it in case an easy hack comes to mind from that work.