The future of content filtering (declarativeNetRequest, Manifest v3, and beyond)

[joey04]

This is both an FYI and some follow-on questions.

The FYI -- For those who haven't yet heard, Google is planning a severe reduction in extension capability for blockers and other content filters. Their current draft of changes includes deprecating the webRequest API and replacing it with declarativeNetRequest. This will only permit a fixed number of purely declaritive rules in a single json list. Moreover, the browser will have final say on what requests get blocked or redirected.

For more info, see this uBlock Origin thread. It has all of the relevant links.

I have two follow-on questions:

1. Is there any interest from the developers here to integrate a filtering engine? I realize this is a big request, but the new API will eliminate the possibility of uBlock Origin, uMatrix, and other robust extensions. Since this project recommends those on the wiki, it will impact this project and its users. My idea is to modify Brave's engine, which is already integrated but has only AdBlock Plus level functionality.

2. If not, can you recommend how to undertake such an effort? I have experience working with large C++ codebases, but nothing browser-related. Any tips, gotchas, etc. would be appreciated.

[SeaKhaki]

I disagree with this notion purely based on the fact adoption is what would make this proposition a profitable endeavour. All I really want to know is whether this would be worth my while to support.

This input comes from the community both of developers (Chromium, extension) and users who would be needing to make the switch.

[RFLEpicGamer]

The mv2 sunset page has been updated for clarity. Until 2024, it should be trivial (I hope) to patch the enterprise policy to always allow mv2 extensions to run. They haven't specified so I'm not certain which one yet. Users could also locally set the relevant policy (on Windows, it could be as simple as a registry key at HKLM\Software\Policies\Chromium). This should suffice until they fully remove the policy and make maintenance require more work. Similarly, for extensions that can comfortably work around the other mv3 restrictions such as arbitrary code execution ― and only require the blocking webRequest API, this can also be solved with patching or setting the relevant policy (ExtensionInstallForcelist). They are not removing the API but restricting it to a user-defined list of extensions, so developers should be able to port their extensions to mv3 and assume blocking webRequest still works as long as users set the policy. I hope this helps.

Just saw this:

* https://www.ghacks.net/2022/09/29/google-delays-the-death-of-manifest-v2-extensions-to-2024/

I thought that it was originally supposed to go out in 2023, so the deadline appears to have been extended a bit. A welcome change. More time to decide on a decision...

All existing Chrome extensions that rely on Manifest V2 will be changed to unlisted.

This means that the chrome extensions that is manifest v2 will be removed from the store more in the future instead of january 2024 or it's the same date?

[dm17]

Good news, for now: Pausing Manifest V2 phase-out changes Discussion: https://news.ycombinator.com/item?id=33925735

Would like to hear your thoughts on a somewhat crazy idea: what if we reimplemented ublock into chromium itself and just had to merge with each new chromium release? A bit of offense could go a long way especially because we're more agile than Google, who are the attackers in this case. Yes it sounds like overkill, but could someone familiar with the code base figure out a clever way to implement content filter inside of chromium? It seems this approach could work around the security issue summarized here: https://forum.level1techs.com/t/the-self-bettering-de-google-experience-grapheneos/150870/19 ?

[SeaKhaki]

I don’t recommend this as it would be easier to just allow extension developers to continue using MV2, at this point since google has postponed introduction it’s something I’d rather leave on a back burner anyways

[SeaKhaki]

On top of that it would make telemetry type transactions an integrated part of the browser, we don’t want that.

[Keirnoth]

Welp, you folks said you are going to do something about it when "Google says they're going to do it."

Well, now they are:

https://arstechnica.com/google/2023/12/chromes-next-weapon-in-the-war-on-ad-blockers-slower-extension-updates/ https://arstechnica.com/gadgets/2023/11/google-chrome-will-limit-ad-blockers-starting-june-2024/

https://developer.chrome.com/blog/resuming-the-transition-to-mv3/

The rollout will be "slow", and I'm praying that enough people yell at them to just call this stupidity off altogether, but it now sounds like they're hellbent on pushing this out since they're trying so hard to stop extensions like uBlockOrigin from working.

Do you guys have any ideas on whether you plan on preserving Manifest v2 in Chromium like Mozilla is or finding some sort of workaround to this? Perhaps working with the UBO guys to permanently implement their ad blocker into your browser? I know that expands the scope of the browser but Manifest v3 is definitely bad for us as an end user since it gives Google way too much control over the availability of extensions. The fact that they're rolling this out claiming it is to "protect our privacy" is IMO quite ludicrous.

[PF4Public]

Perhaps working with the UBO guys to permanently implement their ad blocker into your browser?

I've sent email to @gorhill, but didn't get any reply, perhaps the message could be lost somewhere inbetween or Raymond had no time to answer.

[r7l]

Maybe it would be an option to keep Manifest v2 with some store system similar to what F-Droid is for Android. An opensource alternative to the Google owned playstore.

[dm17]

Perhaps working with the UBO guys to permanently implement their ad blocker into your browser?

I've sent email to @gorhill, but didn't get any reply, perhaps the message could be lost somewhere inbetween or Raymond had no time to answer.

That would be epic... a UBO built into chromium would be able to satisfy several issues. For example, the GOS/Vanadium developers don't allow adblocking for security reasons that a built-in adblocker would solve. A chromium forks collaboration might be skillful as well: https://github.com/ungoogled-software/ungoogled-chromium/discussions/1395

[thestinger]

For example, the GOS/Vanadium developers don't allow adblocking for security reasons

System wide adblocking works fine on GrapheneOS with an app like RethinkDNS and I'm unsure what you mean by us not allowing it. Chromium on mobile doesn't support extensions and the existing extension model is a massive security disaster. The way Apple did things on iOS is a lot nicer but Manifest v3 is a long way from getting rid of all the extremely problematic parts of it.

[gorhill]

working with the UBO guys to permanently implement their ad blocker into your browser

The reality is that I just can't take on more projects than what I already have on my plate.

[senposage]

I mean the end game is to not relie on googles extension store AT ALL which means maintaining your own store or. side-loading extensions manually

the debate about how bad manifestV3 really is is irrelevant .. and its bad if you ask me

but taking any and all influence google has out of the browser is a good thing

and why would this project follow googles rules when it exists to ungoogle- chromium

[senposage]

if ungoogled is interested in a ready made adblock solution why not kang braves? https://github.com/brave/adblock-rust

[gothmog123]

another possibility is the adblock engine from the adblock plus browser. uazo from cromite has already integrated it into his bromite fork, maybe a collaberation with him would be fruitful

[senposage]

we are sliding off topic slightly

really the maintainers need to get together and decide if they are going to follow googles convention and drop manifest V2 or support it side-by side with manifest v3 or stick with manifest V2 or create there own manifest.X

but consensus needs to be reached sooner rather then later because either way this is going to take a bunch of work

./goes back to lurking

[dm17]

@senposage a solution for adblocking that works as well as it does currently is a related consideration of v3 is used, right? As I understand it v2 would be harder to maintain. I'm unsure of the tradeoffs considering UBO could be built into UG.

[Ahrotahn]

Apologies for the late response, this is mostly because we still haven't settled on a late-term plan.

The near-term plan is to introduce a patch that keeps manifest v2 available by default when the deprecation rolls out. This will buy us another year after June.

The long-term plan depends on what actions outside parties make, so it looks like we'll have to be reactive rather than proactive.
It should be feasible to maintain a patch to keep a subset of manifest v2 available, but if Google makes significant enough changes after the full removal then that may not work out.

If that ends up being the case then including patches to integrate an adblocker may be an option, but it would need to be something that could be easily maintained. If the changes needed to do that end up being more complicated than our safebrowsing patches then we may need to work out an alternative option.

[cpuuntery]

if ungoogled chromium in the future plan to remove mv2 support after google remove the code completely from the chromium repo

these two things must be implemented if possible to cover the loss of mv2

1- native ad blocker must support scriptlets. (scriptlets support is very important to trick the page to think that ad ads are not blocked and is a must-have to block youtube ads) and applying custom CSS style

2-support for User Scripts (Violentmonkey has 1 million installs and Tampermonkey has 11 million installs) and there is a website the designed just to share user scripts greasyfork

and from my understanding chromium that manage extention and network request is the same code on Windows, android, and Linux

both Bromite and Cromite support both native ad blocker and User Scripts, so their code is a good starting point if ungoogled chromium went the native route

[cpuuntery]

But if ungoogled chromium in the future plan to continue mv2 support after google remove the code completely from the chromium repo

then i know of only one open source chromium fork the plan to continue mv2 support after google remove the code completely from the chromium is supermium

[crabdancing]

Yeah, so this issue is about to become a lot more relevant. ;)

[deepfold9118]

Apologies for the late response, this is mostly because we still haven't settled on a late-term plan.

The near-term plan is to introduce a patch that keeps manifest v2 available by default when the deprecation rolls out. This will buy us another year after June.

The long-term plan depends on what actions outside parties make, so it looks like we'll have to be reactive rather than proactive. It should be feasible to maintain a patch to keep a subset of manifest v2 available, but if Google makes significant enough changes after the full removal then that may not work out.

If that ends up being the case then including patches to integrate an adblocker may be an option, but it would need to be something that could be easily maintained. If the changes needed to do that end up being more complicated than our safebrowsing patches then we may need to work out an alternative option.

Well, I think it's time you DO settle on a plan. https://www.techradar.com/computing/chrome/googles-manifest-v3-will-make-the-fight-against-third-party-adblockers-even-worse

[networkException]

There's still an enterprise policy for v2 support which we are already enabling internally by default.

To reiterate, settling on a plan isn't really possible here without knowning what Google will do.

[crabdancing]

@networkException That theoretically is only good until 2025 tho. Counterfactual assurances seem maybe-possible, and would probably help the ecosystem know what to expect from ungoogled-chromium.

I mean, if you want to you could assure your users that e.g., if the v2 code is removed completely upstream in such and such scenario, you'll try to keep it functional on this fork beyond 2025. That would probably help inform the maintenance decisions of projects like uBlock Origin too, if they had some assurance that some fork would keep v2 for Chromium. IDK if that's feasible or desirable given your goals, but it would be cool if possible. I assume that the fact you haven't already done it means it might be really difficult, though. Totally understandable.

Edit:

Actually, on the subject of integrating an adblock system, the Angelfish project has had good success integrating adblock-rust, by the Brave browser team. Using Rust is a good choice as it mitigates attack surface for memory exploits, so if there's a plan to integrate a replacement for adblock extensions, that might be worth looking into. This was done awhile back, but I think it's still instructive if ungoogled-chromium wants to take on that challenge.

Rust is also really easy to work with for new features without lots of bugs creeping in, so forking adblock-rust to e.g. add scriptlets probably wouldn't be too hard.

[networkException]

I can only be cautiously optimistic that we'll actually be able to maintaing something like that but its something that we want to see in the project just as anybody else.

I've also been briefly talking to an experienced downstream maintainer who considered contributing nessesarry changes. Still, if Google decides to rip out all the pieces we probably have to drop support.

In general I'd like to avoid introducting ad block support by default unless everything else fails, but its certainly a path this could take in the longer term.

[crabdancing]

I've also been briefly talking to an experienced downstream maintainer who considered contributing nessesarry changes. Still, if Google decides to rip out all the pieces we probably have to drop support.

I guess another option in that scenario is, if the static rules limitations upstream are arbitrary constraints being set in a couple lines of code, you could also have a slightly non-compliant version of MV3 that allows more static site rules -- and then specific projects like uBlock Origin could elect to take advantage of that to offer better adblocking on ungoogled-chromium. I'm hoping uBlock Origin peeps still bother to maintain MV2 support, and they might since Firefox is keeping it, but if not, maybe we could negotiate or set a precedent for a 'MV3+'.

The hope being that it would eventually be a slightly altered spec shared between Firefox, ungoogled-chromium, uBlock Origin, et al. Something backward compatible with normal MV3 but including a few 'extras' bolted on to mitigate Google's user-hostile engineering.

[stephenhawk8054]

The number of static rules is not the only limitations, and actually, that is not uBO Lite's concern either.

The concerns are

• Filtering capabilities not available in MV3
• Declarative nature of MV3 limits custom lists and rules, and can't have fast-update lists. Workarounds can lead to unreliable filtering.

[senposage]

would making a modification of manifest v3 its self to restore capablity be something that should be considered ? something like manifest v3+ perhaps even spoofing the manifest level

[rany2]

What's the point of keeping MV2 support when Chromium drops it completely? I doubt that uBO or other extensions are going to keep supporting MV2 on Chromium after it's killed off officially.

Also, IIRC Mozilla plans to address the issues in MV3 and make it on par with MV2 in terms of capabilities for content blockers like uBO but I doubt that Chrome will take in those changes for their implementations. So there really won't be a reason for uBO to maintain an MV2 version.

Obviously we'll need to see how this plays out.

[lsakdf]

What's the point of keeping MV2 support when Chromium drops it completely? I doubt that uBO or other extensions are going to keep supporting MV2 on Chromium after it's killed off officially.

most browser extension apis do not work properly in mv3. A good example for that would be chrome.alarms - it is almost impossible to use it. It was not easy to develop extensions before but now it got much harder to do so. Because there are a lot of bugs and weird behaviors. For example you are not able to access defined variables in devtools when you debug a service worker. if you type in devtools something like chrome.alarms.onAlarm.hasListener( on.alarm ) it will tell you that on.alarm is not defined.

mv3 is needed to one day remove support for extensions completely like they did in android version of the browser. there are no upside to it.

if ungoogled chromium will be the browser that supports mv2 it will make it much more popular alternative to other chromium forks.

[gamelaster]

@lsakdf this sounds great, and would be ideal situation to keep mv2, but from developer side, with every update and "breaking change" of parts which mv2 uses, it will get harder and harder to maintain, resulting in too much effort, also cause of bugs, etc. So as @rany2 and other said, we should wait an observe.

[railgun77]

@networkException Why almost no one mention the supermium browser ? You can take the code from supermium and merge it with ungoogled chromium fun fact, even supermium uses some of ungoogled chromium code in fact 80% of chromium code is platform-agnostic for the time being, can the ungoogled chromium maintainers please just comment their thoughts about supermium browser

[networkException]

I personally think supporting software for ancient and vulnerable platforms is just ignoring reality and waiting for things to backfire, there, that's the maintainer's comment.

On a more technical note, if there are patches available that are licence compatible with us we will of course look into integrating those. To reiterate, we don't know what will come and especially what kind of resources will be needed to maintain support.

[railgun77]

@crabdancing it doesn't matter what programming languages the ad blocker is written in what makes the ad blocker good. is its ability to parse adblock lists and brave so-called "great" rust ad blocker is not as good as ublock at parsing ad block lists and what about userscripts in your mind, browsers should only support subpar adbloking using "rust" and call it a day it seems that you do not use userscripts, but I assure you there are millions of people who use userscripts

[PF4Public]

for the time being, can the ungoogled chromium maintainers please just comment their thoughts about supermium browser

It is lagging behind Chromium releases significantly, which would make it ultimately too much effort to use their patches.

[crabdancing]

@railgun77

"it doesn't matter what programming languages the ad blocker is written in"

It absolutely does, if you care about performance & memory safety. Rust/Go are ideal options for adblocking business logic, in terms of performance. JS/TS are reasonable from a performance standard (though JS is definitely not ideal from a soundness perspective, for its lack of correctness focus).

You wouldn't write an adblocker in Python, if you want it to be able to keep up with a modern website. You could write it in C, but it's going to be swiss cheese if you do. In general, if you want both as much performance (so it can run on older hardware) and safety as possible, Rust is an ideal language for this application. I agree that their adblocker is inferior in terms of its capabilities, but if we're talking about baking an adblocker into the engine itself -- you're going to add a hell of a lot of memory bugs if you're not doing it in Rust, as opposed to C/C++. And no matter what option you pick, you're probably not going to have it work well enough out of the box without patching it with extra functionality -- which means extra attack surface. A correctness focused language is ideal for this use case.

[dm17]

It would be pretty epic to see a collaboration between the ublock origin devs and UG... But as you say, ublock is mostly JS. However, it has been the premier adblocker for a long time now. Surely they've a lot in mind that they could have / would have done if they had a browser-level rather than extension-level implementation.

[nl255]

I've also been briefly talking to an experienced downstream maintainer who considered contributing nessesarry changes. Still, if Google decides to rip out all the pieces we probably have to drop support.

I guess another option in that scenario is, if the static rules limitations upstream are arbitrary constraints being set in a couple lines of code, you could also have a slightly non-compliant version of MV3 that allows more static site rules -- and then specific projects like uBlock Origin could elect to take advantage of that to offer better adblocking on ungoogled-chromium. I'm hoping uBlock Origin peeps still bother to maintain MV2 support, and they might since Firefox is keeping it, but if not, maybe we could negotiate or set a precedent for a 'MV3+'.

The hope being that it would eventually be a slightly altered spec shared between Firefox, ungoogled-chromium, uBlock Origin, et al. Something backward compatible with normal MV3 but including a few 'extras' bolted on to mitigate Google's user-hostile engineering.

If the goal is to make it work with uBO then I am not sure it is worth the effort as they are apparently not interested. Good luck even trying to work with them or even getting any information out of them about what would need to be added as every time they were asked if there was anything else that they needed and was missing besides the old webRequest API they just automatically post a link to the limitations of declarativeNetRequest, which is obviously irrelevant.

[stephenhawk8054]

gorhill simply won't have time for another project. He has personal life outside github too. There's already a ton of issues and feature requests waiting to be addressed inside uBO itself (https://github.com/uBlockOrigin/uBlock-issues/issues/3151).

[gamelaster]

"Forcing" Manifest V3 in Google Chrome began...

https://github.com/uBlockOrigin/uBlock-issues/wiki/About-Google-Chrome's-%22This-extension-may-soon-no-longer-be-supported%22

[PF4Public]

"Forcing" Manifest V3 by Google began...

Is it a screenshot from ungoogled-chromium?

[gamelaster]

@PF4Public no, it's from Google Chrome, sorry, I should have say that in the comment. Anyway, what I want to say is, that Google is now actively and publicly deprecating Manifest V2 extensions, so more breaking changes can happen soon also in Chromium codebases.

[PF4Public]

I recently noticed some websites breaking spectacularly with noscript enabled in ungoogled-chromium. Not sure if it has something to do with V2 deprecation, didn't have time to investigate. :(

[DianaNites]

For the record, uBlock Origin has officially declared they WILL keep supporting Manifest V2

https://github.com/uBlockOrigin/uBlock-issues/wiki/About-Google-Chrome's-%22This-extension-may-soon-no-longer-be-supported%22

Will development of uBO continue?

Yes, there are other browsers which are not deprecating Manifest v2, e.g. Firefox.

https://x.com/gorhill/status/1819394129618428169

[TCB13]

For the record, uBlock Origin has officially declared they WILL keep supporting Manifest V2

https://github.com/uBlockOrigin/uBlock-issues/wiki/About-Google-Chrome's-%22This-extension-may-soon-no-longer-be-supported%22

Will development of uBO continue? Yes, there are other browsers which are not deprecating Manifest v2, e.g. Firefox.

https://x.com/gorhill/status/1819394129618428169

They seem to be releasing a light version of the extension with support for v3... question: how's thing going to play out for Ungoogled? Will they still release a chrome compatible v2 version in the future? Will we lose the extension entirely?

[PF4Public]

how's thing going to play out for Ungoogled? Will they still release a chrome compatible v2 version in the future? Will we lose the extension entirely?

It is a good question to ask… in their repository and back-link here so that we are all informed of their answer.

[jstkdng]

since brave is saying they will continue supporting ad blocking, maybe a new project could be made to "unbrave" brave browser? Brave browser without adds, crypto, phoning home. Maybe call it fear-browser.

[nikkehtine]

@jstkdng uhhh... Let's maybe stick with "Lion Browser" 😅

[crabdancing]

@jstkdng I vote for Coward.

[jstkdng]

@nikkehtine let's try not to be related to their IP to prevent any legal trouble. It should be an antonym or the opposite.

@crabdancing i also thought that but it's too crude imo. Maybe hyena browser, since we'll be leeching off of brave :P

[crabdancing]

@jstkdng Oooh, I was memeing but I unironically quite like Hyena :3