Chrome extension manifest v3 proposal

[greghuc]

Description

This issue is a heads-up on the proposed Chrome extension manifest version 3, which will have a significant impact on ad-blockers.

There is a tracking bug at: https://bugs.chromium.org/p/chromium/issues/detail?id=896897

In-progress design doc: https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3NzzhHzc-qnk4w4PX-0XMw8/edit#

Might be worth a review, and giving feedback via the tracking bug.

regarding all dev. on dnr - https://bugs.chromium.org/p/chromium/issues/list?can=2&q=declarative+net+request

"Migrating to Manifest V3" (timeline) - https://developer.chrome.com/extensions/migrating_to_manifest_v3

[uBlock-user]

Yes this was announced back in October - https://blog.chromium.org/2018/10/trustworthy-chrome-extensions-by-default.html

Not much for uBO to do though..

[jspenguin2017]

I'm not sure how are they planning to change the background process and origin access, it might cause some issues. Beside that, I don't see anything else that can cause problems. As for remotely-hosted code, it really should've been forbidden since the beginning.

[joey04]

It appears that Google is phasing out the webRequest API in favor of their new declarativeNetRequest.

The current Manifest V3 draft states:

The declarativeNetRequest API is an alternative to the webRequest API. At its core, this API allows extensions to tell Chrome what to do with a given request, rather than have Chrome forward the request to the extension. Thus, instead of the above flow where Chrome receives the request, asks the extension, and then eventually gets the result, the flow is that the extension tells Chrome how to handle a request and Chrome can handle it synchronously. This allows us to ensure efficiency since a) we have control over the algorithm determining the result and b) we can prevent or disable inefficient rules. This is also better for user privacy, as the details of the network request are never exposed to the extension.

This API is currently being implemented, and will be available to both the current manifest version and Manifest V3, but will be the primary way to modify network requests in Manifest V3.

Looking at the new API page, it's a rather different way of doing things. I would expect big changes needed to uBO and other blockers. (Plus there's the change from background page to ServiceWorker.)

[uBlock-user]

Another change would be dynamic Content Scripts similar to what Firefox has where extensions can inject contentScript and have it execute before page finishes loading.

[gwarser]

declarativeNetRequest

Basic ABP-like syntax, 30000 filters MAX...

[uBlock-user]

That was declarativeWebRequest, that was shelved.

[gwarser]

I don't see limits to the number of the rules in declarativeWebRequest. declarativeNetRequest have better syntax, but number of rules is limited.

[gorhill]

It appears that Google is phasing out the webRequest API

It is explicitly stated this is meant to phase out webRequest API? If so, where? Quickly perusing the page, I don't see such phase out being stated. If there is really such phase out planned, and if the declarativeNetRequest is strictly an implementation of static filtering à la ABP, this would be the death of uBO and uMatrix.

There is no way to transpose either either dynamic filtering, dynamic URL filtering, per-site/per-scope switch logic (let's refer to all these as "dynamic filtering"), into static filters. Dynamic filtering logic requires an arbitrary amount of block/allow rules overriding other block/allow rules based on specificity. There is no concept of specificity in static filtering -- and even more, there is no concept of dynamic filtering rules relinquishing filtering to static filters (dynamic filtering's noop rules).

Basic ABP-like syntax, 30000 filters MAX...

And there I was recently testing how uBO handled over half a million network filters with the 3rd-gen HNTrie...

Issues of performance and privacy lie with web sites, not uBO -- so I don't feel concerned with the issues of privacy and efficiency being put forth as advantages of using declarativeNetRequest over webRequest.

[uBlock-user]

https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3NzzhHzc-qnk4w4PX-0XMw8/edit#heading=h.ypclvihky0p6

Read the paragraph that's titled DeclarativeNetRequest. They will keep webRequest API but it will not block, modify or redirect anymore, that will be handled by NetRequest API with v3.

They do plan to discourage the use of webRequest API through with its new replacement - https://docs.google.com/document/d/1nPu6Wy4LWR66EFLeYInl3NzzhHzc-qnk4w4PX-0XMw8/edit#heading=h.xe5njuo7voeb

[uBlock-user]

30000 is mentioned in https://developer.chrome.com/extensions/declarativeNetRequest#property-MAX_NUMBER_OF_RULES, not sure what to make of this, can you shed some light there ? I don't think it is related to 30K ABP styled filters.

[gorhill]

Look at "Rule", it's clearly made to implement ABP-compatible filters:

'|' : Left/right anchor: If used at either end of the pattern, specifies the beginning/end of the url > respectively.

'||' : Domain name anchor: If used at the beginning of the pattern, specifies the start of a (sub-)domain of the URL.

'^' : Separator character: This matches anything except a letter, a digit or one of the following: _ - . %. [and so on...]

[joey04]

This new API is even worse than being restricted to 30k ABP-style rules -- the rules must also be in a single, bundled json file. Thus any rule changes means a full extension update. (Blockers restricted to this API would be "static" in every way.)

Thinking big picture, it's not that surprising. Last year Google started to bundle a lightweight ad blocker in Chrome, and I recall thinking that something like this would happen. On the bright side, perhaps this could be a boon to Mozilla, assuming they don't foolishly neuter their API in the same way.

[Kusresa]

It is still a draft but is really a big downgrade if it stays this way. Looks to me like Google is doing this to reduce the amount of damage malicious extensions could do using this API by severely limiting the existing one and calling it a privacy and efficiency win....while of course giving more control to Google over what ads/trackers/requests can be blocked. This affects trusted and more advanced extensions like uBO the most.

Hopefully Mozilla and other browser makers don't follow suit on this..

[jspenguin2017]

declarativeNetRequest in its current state is really underpowered and isn't going to be enough. However, assuming that they will implement a way to modify the rules and the 30k limit will be raised or removed, it looks like the only thing missing is RegExp rules.

Since we will be able to dynamically modify content scripts, we can still implement scriptlets and cosmetic filtering properly.

[uBlock-user]

It's still not set in stone, so lets see where it goes.

[gorhill]

It's still not set in stone

The fact that they are planning to remove a proper blocking webRequest API with no word of an equivalent replacement is a sign of intent, that is, reducing the level of user agency in their user agent (aka Google Chrome).

How to do this? Use privacy/performance as Trojan arguments to rationalize reducing user agency over what all bloated web sites throw at people's user agents. That new declarativeNetRequest API seriously reduces what blockers can do, to the point they will become distinguishable only by their UI, not their capabilities. As a user, I personally wouldn't accept browsing the world wild web without the advanced features in uBO, I find this unthinkable.

There are no issue of privacy/performance with uBO, rather the opposite by giving back to users the power of clamping down on what web sites throw at them, so that argument is just plain fallacious as far as uBO is concerned.

Chromium got its webRequest API at a time it was trying to gain market share against Firefox (Sep 2011), where Adblock Plus, Ghostery, Disconnect, NoScript, and other such extensions were the most or among the most popular extensions on Firefox.

I don't expect Firefox to follow suit and also deprecate its own webRequest API.^[1] I am confident uBO will still exist on Firefox.^[2]

---

[1] Actually Firefox's own webRequest API is better designed as it's possible to return a Promise, which makes it possible to defer returning an answer to some point in the future.

[2] Which is already better equipped than Chromium's version of uBO -- example, example -- (and also better equipped than the Firefox legacy version).

[uBlock-user]

is a sign of intent

Yes, but they're not doing this specifically targeting uBO, other blockers will be affected by this too, won't they ? So I still think it won't end up like this. Like you said, it will be the death of these two extensions, why would they bother doing that after all this time ? If they wanted to kill uBO, they could have done this years ago by deprecating APIs or limiting them.

[gorhill]

I'm not saying they are targeting uBO specifically, more that they are targeting the capabilities expressed in uBO/uMatrix.

What is being said now is that the maximum capabilities content blockers will be allowed come down to a maximum of 30,000 ABP-compatible filters.

Even without dynamic filtering and per-site switches, etc., uBO already enforces over 90,000 filters (which themselves go beyond ABP filter syntax) with just the default filter lists, not counting the regional one which may be activated automatically at first install, and other commonly used ones such as Fanboy Social. When I select only EasyList, uBO reports 42,000 network filters, so even EasyList alone won't be enforceable with the declarativeNetRequest API.

Beside the low maximum of 30,000, ABP-compatible filters have no sense of specificity, hence why dynamic filtering can't be implemented with such approach (and if they did it would be a pain to have to recompile the whole filtering profile when merely adding/removing a rule/switch through a click). Also, this makes it impossible to implement important filter option, which purpose is to override exception filters.

So I still think it won't end up like this.

My comments are made with what is being said now with regard to manifest v3.

[androidacy-user]

Why would Google want to be compatible with adblocking? After all, their business is advertising. Not saying that is would be right, but it's not unexpected

---

From: Raymond Hill notifications@github.com Sent: Monday, January 21, 2019 2:00:46 PM To: uBlockOrigin/uBlock-issues Cc: Subscribed Subject: Re: [uBlockOrigin/uBlock-issues] Chrome extension manifest v3 proposal (#338)

I'm not saying they are targeting uBO specifically, more that they are targeting the capabilities expressed in uBO/uMatrix.

What is being said now is that the maximum capabilities content blockers will be allowed come down to a maximum of 30,000 ABP-compatible filters.

Even without dynamic filtering and per-site switches, etc., uBO already enforces over 90,000 filters (which themselves go beyond ABP filter syntax) with just the default filter lists, not counting the regional one which may be activated automatically at first install, and other commonly used ones such as Fanboy Social. When I select only EasyList, uBO reports 42,000 network filters, so even EasyList alone won't be enforceable with the declarativeNetRequest API.

Beside the low maximum of 30,000, ABP-compatible filters have no sense of specificity, hence why dynamic filtering can't be implemented with such approach (and if they did it would be a pain to have to recompile the whole filtering profile when merely adding/removing a rule/switch through a click). Also, this makes it impossible to implement important filter option, which purpose is to override exception filters.

So I still think it won't end up like this.

My comments are made with what is being said now with regard to manifest v3.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/uBlockOrigin/uBlock-issues/issues/338#issuecomment-456171375, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Aq69FBPVyGJz_6AVJU3Oo0hj8_MjQJNJks5vFg5egaJpZM4ZRT_H.

[uBlock-user]

At this juncture, assuming it will get implemented, can you do something about it like not updating to v3 ? or should Chromium users be ready to abandon ship for good ?

[gorhill]

should Chromium users be ready to abandon ship for good ?

I won't tell people what to do. I am pointing out that removing the blocking ability of the webRequest API means the death of uBO, I won't work to make uBO less than what it is now. I quote the document (my emphasis):

WebRequest

Summary

In Manifest V3, we will strive to limit the blocking version of webRequest, potentially removing blocking options from most events (making them observational only). Content blockers should instead use declarativeNetRequest (see below). It is unlikely this will account for 100% of use cases (e.g., onAuthRequired), so we will likely need to retain webRequest functionality in some form.

Currently uBO uses blocking listeners on both webRequest.onBeforeRequest (to implement static-, rule-, switch-based network filtering) and webRequest.onHeadersReceived (to implement disabling or restricting JS execution, static filters' csp= options, large media filtering, and other filtering capabilities).^[1]

With this information on hands, everybody is free to decide for themselves.

---

[1] There are four listeners in the webRequest API which can be used in blocking mode: onBeforeRequest^{(uBO, uMatrix)}, onBeforeSendHeaders^(uMatrix), onAuthRequired, onHeadersReceived^{(uBO, uMatrix)}. uBO uses two of them, uMatrix uses three of them.

[androidacy-user]

Hey, I haven't used Chrome in years. I use waterfox

---

From: Raymond Hill notifications@github.com Sent: Monday, January 21, 2019 2:37:02 PM To: uBlockOrigin/uBlock-issues Cc: colbycdev; Comment Subject: Re: [uBlockOrigin/uBlock-issues] Chrome extension manifest v3 proposal (#338)

should Chromium users be ready to abandon ship for good ?

I won't tell people what to do. I am pointing out that removing the blocking ability of the webRequest API means the death of uBO, I won't work to make uBO less than what it is now. I quote the document (my emphasis):

WebRequest Summary

In Manifest V3, we will strive to limit the blocking version of webRequest, potentially removing blocking options from most events (making them observational only). Content blockers should instead use declarativeNetRequest (see below). It is unlikely this will account for 100% of use cases (e.g., onAuthRequired), so we will likely need to retain webRequest functionality in some form.

Currently uBO uses blocking listeners on both webRequest.onBeforeRequest (to implement static-, rule-, switch-based network filtering) and webRequest.onHeadersReceived (to implement disabling or restricting JS execution, static filters' csp= options, large media filtering, and other filtering capabilities).

With this information on hands, everybody is free to decide for themselves.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/uBlockOrigin/uBlock-issues/issues/338#issuecomment-456179825, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Aq69FEkGLSdowCeR1Swc7DCNFOzIFZQ_ks5vFhbegaJpZM4ZRT_H.

[Kusresa]

I think the best thing people can really do for now is to get the word out to extension developers and browser developers (especially Google) that the proposed APIs and manifest should not be restricted to such an extent and that users should retain enough freedom and capabilities to easily control what to do with extensions and requests within their browser.

Once the requests API in the manifest v3 proposal is set in stone and implemented it will be too late of a surprise for the majority of unaware extension users who will notice a shifting of how and what ads/trackers/requests get blocked and it will be near impossible to rollback the changes as the browser market leader has a low incentive to do so.

I don't want to sound too dramatic but the implementation of the requests API in the manifest v3 proposal as it is right now could be the beginning of something that will have wider implications on the web and users' ability to decide how they can browse it. Due to Google's position of power on the web and influence on websites it will almost certainly affect more than just Chromium/Chrome users.

[uBlock-user]

get the word out to extension developers and browser developers (especially Google

Post here and let them know -- https://bugs.chromium.org/p/chromium/issues/detail?id=896897

[grumpygeek]

Neither Otter nor Falkon are based on Chromium so they might be options. Right now neither supports user added extensions, but maybe one or both will eventually. Even new Firefox is questionable. Unfortunately, you can't follow the crowd; best alternatives are the ones that don't have measurable market share.

[max-hk]

Is it possible to implement more than 30,000 filters by installing more than one instance of ublock orign, like 2 or 3 same extension in different filters and ID?

[uBlock-user]

comment section there is locked now, so the new place is at- https://groups.google.com/a/chromium.org/d/topic/chromium-extensions/veJy9uAwS00/discussion

[patrickdrd]

can't we star it?

[uBlock-user]

Starring is always possible, however if you want to add your voice/concern, you will have to do it in the google group link from now onwards.

[patrickdrd]

I would like to star "Manifest V3: Web Request Changes" but I can't see an option to do so, only replies I guess

[mapx-]

you can click the arrow on the right of "10 posts by 9 authors" and click "email updates"

[patrickdrd]

I can't see that option: https://imgur.com/ZAg3csh

[mapx-]

sure, do you have a google account ? use it to log in

[patrickdrd]

ok, I logged in, I joined group and suddenly it started raining emails...

too much for a draft change, I left the group again...

in other words too much panic so soon I guess..

let's wait and see

[mikhoul]

Hi @gorhill ,

I use Nano but since both Nano and UBO extension are almost the same I was asking to @jspenguin2017 if forking Chromium to integrate natively UBO and Nano to bypass the weakned API would be possible.

So today when browsing reddit I've seen that Brave browser is already doing it:

A developer on the Brave team was explaining it on Reddit:

---

_Bat-chriscat BAT TEAM[M] 34 points 1 day ago* RemindMe!

It's worth noting that our Brave Shields (ad blocker) is not an extension; it is natively implemented. So extension API changes leave our shields unaffected.

Edit: We can always remove any code or update we don't like from the Chromium base we use. So even if this didn't just affect extensions but something deeper, we could just exclude it._

---

Source: https://old.reddit.com/r/brave_browser/comments/aijqm4/chrome_may_soon_change_how_3rd_party_ad_blockers/eeopqpb/

I wanted to share this possible solution with you and to know what do you think about it ? 🤔

Having a browser based on Chromium fork to integrate both UBO and Nano since you share 98% of the same code seem to be something that would benefit everybody.

Original discussion with @jspenguin2017 : https://github.com/NanoAdblocker/NanoCore/issues/238#issuecomment-456905809

Regards :octocat:

[mapx-]

https://groups.google.com/a/chromium.org/forum/#!topic/chromium-extensions/veJy9uAwS00/discussion

First off, I'd like to reiterate a few points:

• The webRequest API is not going to go away in its entirety. It will be affected, but the exact changes are still in discussion.
• This design is still in a draft state, and will likely change.
• Our goal is not to break extensions. We are working with extension developers to strive to keep this breakage to a minimum, while still advancing the platform to enhance security, privacy, and performance for all users.

Transparency and openness is core to the Chromium project, and we welcome feedback because we think having technical discussions in the open is valuable. However, these discussions should be kept productive, and should always follow our code of conduct.

For feedback specific to details of the declarativeNetRequest API, such as the 30k rule limit, should be raised on the thread that was sent out in November soliciting feedback on that API.

I certainly understand that there are a number of concerns around limiting the power of the webRequest API in favor of a declarative API - it is much more restrictive, and doesn't offer the flexibility the webRequest API does. The most helpful feedback for us is the exact cases that this would impact, their importance, and the reasons they are impossible through either the declarative API or through other extension APIs. A number of these were raised in comment #23 on the bug, including:

• increased flexibility for specificity/overrides of rules
• blocking elements of certain sizes
• disabling JS execution through injection of CSP directives
• removal of outgoing cookie headers

Another issue that has been raised here a number of times is the ability to dynamically and instantly update the ruleset.

Having a list of the use cases that aren't feasible is incredibly useful for us in moving forward.

Please also try to keep discussions on-topic. This is not the right thread for discussions around other aspects of potential Manifest V3 changes, the WebExtensions standardization effort, or non-technical discussions.

Cheers,

• Devlin

[joey04]

Neither Otter nor Falkon are based on Chromium

Incorrect. They're both based on Chromium, using the Qt WebEngine framework that is a wrapper for Chromium.

The reality is there are only 3 browser engine developers left: Google, Apple, and Mozilla. All of the hobby FOSS browsers are based on one of these. There are no other practical options.

I realize this is an emotionally-charged thread given the severity of what Google may do to its API. But let's try not to spread misinformation here.

And I'm glad I raised awareness of this issue here 2 weeks ago. After reading the V3 draft I was quite surprised nobody had posted in detail here. It's good the word has gotten out, and hopefully that will dissuade Google from neutering webRequest. (But there are big money interests at play here, so it certainly won't be just a technical decision.)

[joey04]

Having a browser based on Chromium fork to integrate both UBO and Nano since you share 98% of the same code seem to be something that would benefit everybody.

@mikhoul - you seem to be unclear about what you're proposing.

If Google goes ahead with its draft changes and neuters webRequest, it will effectively end uBO for Chromium, as gorhill has described above. (Keep in mind they already have declarativeNetRequest implemented in Chrome beta. These are not just words on a document.)

At that point, what to do if we end-users want to retain powerful content filtering capability?

There's still Firefox, though its future isn't exactly rosy. Its market share continues to decline and web compatiblity problems may creep in, which would be a serious problem for its viability.

Another option would be a new FOSS fork of Chromium that leverages the Brave filter engine. (To their credit, Brave shares it as MPL, so others can use it.) Brave itself has their own Basic Attention Token ad vision, so I personally wouldn't use it; I'm too accustomed to having lots of control via uBO.

But a browser fork is a massive effort. Much bigger than uBO or any other extension. For those who don't know, web browsers are huge programs with millions of lines of code. Most of it is very technical stuff that paid developers from the 3 companies I already mentioned take care of: rendering, JavaScript engine, the many libraries needed for image and video decoding, and all the other things a browser does.

I think it'd be great if such a project arises, and I have some C++ experience so I would be willing to help out in some capacity. But it's a very serious effort to keep up with Chromium and maintain patches like a modified version of Brave's built-in filtering engine.

[gorhill]

I found meta issue tracking the development of the declarativeNetRequest API, dated Feb 2017: https://bugs.chromium.org/p/chromium/issues/detail?id=696822.

I am not sure when the declarativeNetRequest API became publicly documented. Google Chrome's own built-in ad blocker went live in Google Chrome 64 on January 2018, and from what I gather the declarativeNetRequest API shares the same code as the built-in blocker (that would be expected).

There is an interesting comment, Feb 13 2018:

So far I have been using the easylist as my source of filters. I use a parser to convert the easylist into a filters.json (attached). This conversion works for 25891/40639 of the easylist resource blacklist filters. Also 4464/5578 of the resource whilelist filters can be converted. The easylist has 18848 element hiding rules which DNR will not support.

So from this I get that the content of EasyList was used to design the declarativeNetRequest API^[1] and the commenter report that a total of 30,355 rules were gathered from EasyList^[2]. This is interesting given that the maximum number of rules allowed by the declarativeNetRequest API has been set at 30,000. Coincidence?

That only EasyList itself was considered explains the limitations of the design with regard to uBO (and AdGuard), given that EasyList maintainers restrict themselves to only work with what Adblock Plus supports.

So far I haven't seen any comments coming from Adblock Plus/AdBlock people. Given that EasyList alone can barely be supported by the new API in one single extension, this does not even leave place for all the exception rules from ABP's "Acceptable Ads". I don't know how many exception filters are in there^[3], but I can count over 14,000 lines in the file, https://easylist-downloads.adblockplus.org/exceptionrules.txt).

---

[1] Even the option genericblock, which is barely used out there (2 instances in EasyList) has been implemented, though it's not exposed to the outside world. I declined supporting that option as I saw it as a anti-user approach to countering anti-blockers.

[2] EasyList probably contains more filters by now.

[3] uBO does not support document option in exception filters, because.

[mapx-]

https://groups.google.com/a/chromium.org/forum/#!topic/chromium-extensions/qNqURIh4Nss

[yourduskquibbles]

@gorhill Not sure if it would be helpful if you need to make a case demonstrating examples of what uBlock Origin empowers the user to do besides adblocking, but my list of CSS Style Modifying Filters[1] that is a sublist of the Web Annoyances Ultralist may be a useful point of reference if you want to provide examples of what the downstream impact of the change would potentially mean for users of your extension.

The primary purpose of the Web Annoyances List & CSS Style Modifying Filters sublist is not blocking ads. Instead it is a list intended only to enhance the user experience on the web through CSS modifications to distracting elements.

[ajsnyde]

Another option would be a new FOSS fork of Chromium that leverages the Brave filter engine.

@joey04 Sorry, could you explain what a "FOSS" fork is?

[gorhill]

FOSS = Free and open-source software

[uBlock-user]

https://groups.google.com/a/chromium.org/d/msg/chromium-extensions/veJy9uAwS00/MZc9Q6nBGgAJ

Once v3 moves to production, there will be a transition period before support for v2 is removed. There is not a firm timeline yet, but we will announce it broadly once there is.

https://groups.google.com/a/chromium.org/d/msg/chromium-extensions/veJy9uAwS00/7Y7jVLTCGgAJ

[okiehsch]

So far I haven't seen any comments coming from Adblock Plus/AdBlock people.

https://groups.google.com/a/chromium.org/d/msg/chromium-extensions/veJy9uAwS00/vIkW_hLCGgAJ

[androidacy-user]

They can also do away with "inefficient" rules How about ensuring their own ads can't be blocked? It's feasible with Chrome handling everything, and they are an ad driven company. I've myself already completely switched to Firefox. Slower, but more privacy and user choices friendly

[uBlock-user]

I am not sure when the declarativeNetRequest API became publicly documented

https://docs.google.com/document/d/1E5bV3nYlj6UvNblk_mG2DYbwFVK-qnl5KnEYHqKpgAM/edit?usp=sharing

Around since Feb 2017 as per c17 in that bug you found on the tracker.

[baraa272]

They can also do away with "inefficient" rules How about ensuring their own ads can't be blocked? It's feasible with Chrome handling everything, and they are an ad driven company. I've myself already completely switched to Firefox. Slower, but more privacy and user choices friendly

use waterfox its same speed as chrome and not bloat like original firefox

[lewisje]

and even less compatible with the modern Web

[baraa272]

anyway adding to this matter its looks like users will have 2 choices either

1. abandon chrome and its forks for good
2. finding a way to implement ublock internally like brave adblocker