Requiring Service Workers for extensions fundamentally user hostile

[ghostwords]

This is related to #44, #11, #25, #39 and #73.

Service Worker-based extensions are Event Pages (persistent: false background pages) made mandatory and with additionally degraded capabilities. Some extensions fit the non-persistent/ephemeral model well. Many do not. Requiring all extensions to become non-persistent appears to be a fundamentally extension developer and user hostile requirement. It violates "user-centered", "compatibility", "performance" and "maintainability" design principles.

The downsides of Service Workers-based extensions include higher development¹ and debugging² complexity, worse performance³, and missing features (no DOM!). The third point in particular gates innovation behind browser developers. Working on an idea that requires Web APIs not supported in SWs⁴? Now at minimum you have to get Chromium and Firefox on board.

Consider also that there are still unaddressed showstopper bugs⁵ as well as inexplicable functionality gaps (such as #93) with SW-based extensions in Chrome.

A Firefox engineer writes:

Background pages will be replaced by background service workers (bug 1578286). This is a substantial change and will continue to be developed over the next few months. We will make a new announcement once we have something that can be tested in Nightly.

What are the benefits of Service Worker-based extensions for extension developers? Is any public discussion available from the Mozilla side?

---

[1] See crbug 1185226 where a whole new, otherwise unnecessary storage API is created in an attempt to fill some of the egregious gaps in the ephemeral model. Also see crbug 1128240 that proposes headless windows (something you already get by default in MV2).

[2] Examples: degraded browser UX, missing tooling.

[3] An overview of the inherent performance problem, see also this crbug with further discussion.

[4] Examples: playing audio, using the clipboard, communicating via WebRTC data channels. In short, everything in Window and Document that's not also in WorkerGlobalScope.

[5] Examples:

• crbug 1024211: Manifest V3: webRequest listeners not called after service worker stops
• crbug 1189678: Chrome browser running a MV3 extension that uses Native Messaging does not respond to messages from the Native Messaging Host after 5 minutes.

[Rob--W]

For visibility, note that the topic of service workers has been raised in two of the WECG meetings (including mentions of some of the bugs and issues that you've described):

• "Service worker topic" at https://github.com/w3c/webextensions/blob/main/_minutes/2021-06-24-wecg.md
• "Issue 11: WebExtensionWorker instead of Service Worker" at https://github.com/w3c/webextensions/blob/main/_minutes/2021-07-08-wecg.md

To answer your question directed to Mozilla:

What are the benefits of Service Worker-based extensions for extension developers? Is any public discussion available from the Mozilla side?

For Mozilla, the primary driver for working on Service workers in MV3 is for cross-browser compatibility. We are aware that some functionality is lost in the background scripts between MV2 and MV3. As our efforts on MV3 progress we intend to look for alternatives and input to retain some of that functionality. We would like to get to a state that works for extension developers and other browser vendors.

We see benefits in SW that they are a standardized execution environment with defined lifetimes by design. We have not settled on a maximum execution time though, given the disadvantages and the fact that there are trivial user-unfriendly ways to work around the limit. Regardless, it will be necessary that extensions are designed to account for the possibility of being suspended, because that matches with reality (we do occasionally see user/dev reports about extensions not working all of a sudden across all platforms). Extensions on Android are currently not run in a separate process (but the main browser process, which is a risk) because extensions on Firefox for Android are not designed to handle the possibility of being terminated by the OS. This is one of the reasons why only a limited set of trusted extensions are supported on Android.

Another aspect of service workers is that unlike web pages, they run on independent worker threads instead of a single main thread. The separation of background threads and other extension pages enables the background script to be responsive even if extension pages (UI) are blocked/busy. An example of a concrete case that I’ve encountered (in the role of an extension developer instead of a browser engineer) is that when I debug a UI page of an extension that uses the webRequest API in the background, that the browser as a whole is unusable because the paused debugger in the main thread prevents the extension from being able to handle network requests. With an independent background worker, this kind of issue goes away. This benefit comes at a cost: several DOM APIs are not thread-safe (notably the DOM parser) and therefore not supported in service workers. Efforts to support that in extensions will benefit the web platform, and vice versa.

[ghostwords]

Thanks for the response!

What prevents persistent extension background pages from becoming a standardized execution environment?

How was the previous version of Firefox for Android able to support all Firefox desktop extensions?

[ghostwords]

After discussing this with several content blocking extension developers, we have decided to implement DNR and continue maintaining support for blocking webRequest.

-- https://blog.mozilla.org/addons/2021/05/27/manifest-v3-update/

How does Mozilla plan on continuing to support blocking webRequest in Service Worker-based extensions?

[Rob--W]

Thanks for the response!

What prevents persistent extension background pages from becoming a standardized execution environment?

The value in the standardized aspect of Service workers is that they are a first-class concept in the web platform, and that available APIs are designed (whether upfront or retroactively) to be compatible with Service Workers.

Just "specifying" persistent background pages or event pages by adding it to the draft spec here does not automatically result in well-functioning web platform APIs. For example, you mentioned the absence of a clipboard API in service workers; despite the API being present, the document.execCommand-based clipboard API used to not work in Firefox's background pages either (now it works, but only with additional engineering efforts). Other examples of commonly used web platform APIs that are still not working in Firefox's background page are popups (window.open) and modal dialogs (alert, confirm and prompt).

How was the previous version of Firefox for Android able to support all Firefox desktop extensions?

( I read this question in relation with my remark "This is one of the reasons why only a limited set of trusted extensions are supported on Android.". If not, please restate your question. )

The previous version of Firefox for Android (Fennec) ran everything (the browser UI, web content and extensions) in a single process, so the concept of separate processes was not relevant. Modern browser architectures use multiple processes for several reasons (including reliability, performance and security), with at least a separation between privileged browser UI and third-party code (web pages). In contrast to Fennec, Firefox for Android (Fenix) now uses multiple processes for web content. As mentioned before, extensions still run in the main browser process, which is not sustainable in the long term.

After discussing this with several content blocking extension developers, we have decided to implement DNR and continue maintaining support for blocking webRequest.

-- https://blog.mozilla.org/addons/2021/05/27/manifest-v3-update/

How does Mozilla plan on continuing to support blocking webRequest in Service Worker-based extensions?

Service workers are not fundamentally incompatible with a blocking webRequest API. The cost and overhead of resuming a suspended worker is a point of concern, which is one of the reasons why we want to carefully examine the lifetime aspects of the service worker.

[Jack-Works]

The value in the standardized aspect of Service workers is that they are a first-class concept in the web platform, and that available APIs are designed (whether upfront or retroactively) to be compatible with Service Workers.

As I pointed out in #11, the pros of Service Workers don't make it automatically the suitable solution to the Web Extension. We need a new dedicated worker type.

[cuylerstuwe]

Live sync (e.g., WebSockets) is a pretty major "elephant in the room" w/ regard to modern web features unsupported by service workers.

So far, the only solutions I've seen suggested for resolving this are a handful of workarounds that essentially boil down to "build what's effectively a copy of background.js".

Examples, examples... everyone likes examples.

OK:

One of the extensions someone has paid me to write in the past was a work-finding tool, to make it easier to find jobs on microtask sites (e.g., think Amazon Mechanical Turk, etc.). This used a crowdsourced data funnel where everyone who was looking for work implicitly contributed the jobs their device had seen, in exchange to see the jobs other devices had seen. After pushing your most recently-seen jobs up to the server, they would be propagated down to all interested parties' clients in the background via websockets. If a client received a job that matched parameters its user was interested in, it would notify the user, so that user could act upon it quickly.

"Quickly" is the operative word here; If you receive a notification about a microtask a minute after it's posted, the only purpose that notification serves is to guilt-trip you about what you missed out on (they go fast!).

I can't think of how we would implement something like this without running all the time; Constant resource drain is kinda the entire point.

I'm worried that the pattern for MV3 extensions is basically going to be to demand of users: "Don't close this tab, or the extension will cease to function"... a lot like what Tile is forced to awkwardly do to work within the confines of iOS:

There are countless similar examples I can think of that just wouldn't fit.

One timely example:

Want a PS5 or GPU without paying a scalper to buy one in 2021?

An MV2 browser extension might be able to help you with that, but an MV3 extension based on service workers is going to have a rougher time.

Some extensions fit the non-persistent/ephemeral model well. Many do not.

I'm a top 1% freelancer on Upwork specializing in browser extensions, and have therefore either built or participated in building countless extensions for individuals, startups, and enterprises.

I can easily say that at least half of what I've been paid to write has no clean transfer to the ephemeral central process we're proposing w/ service workers. I can say from this extensive professional experience that most do not. I imagine that if a persistent model is not provided, most developers who try to make anything more significant than a toy app will end up essentially building background.js... albeit with significant-degraded usability.

[cuylerstuwe]

Another common example for where a persistent background connection is needed in web extensions is this case:

"An extension needs to send/receive data immediately between a desktop app."

In browser extensions currently (in MV2), this is generally achieved either through the NMH (Native Messaging Host) or a WebSocket connection to a localhost server.

Neither NMH nor WebSockets works properly with a Service Worker which isn't artificially kept alive to effectively mimic a persistent background.js page.

This may surprise some, but this need to connect a browser extension with a local tool is surprisingly common. Many browser extensions are themselves extensions of desktop apps into the browser, and expect to be able to communicate with them. 1Password should be just one familiar mainstream example of a browser extension that makes a direct connection with a desktop app.

In the absence of a persistent connection, there's no clear means for bidirectional live communication between a desktop app and a corresponding browser extension. This severely degrades functionality that users have come to expect.

[cuylerstuwe]

To add further onto my last point:

Chrome's teams have justified non-persistent service workers by saying that extensions should be "user-driven".

However, in what seems to be a failure of imagination (perhaps from being so focused on Chrome, rather than on users?), they seem to have overlooked that a lot of "user-driven" behavior occurs outside of the browser.

A couple of examples:

• User updates a password on desktop password manager software. This is linked via Native Messaging Host API to a browser extension's persistent background page, in order to consume the change live. User expects this changed value to be applied immediately within the browser without needing to refresh anything. User-driven? Check.
• User installs an Alexa skill or uses some other alternative input method meant to drive Chrome. This ultimately communicates to a browser extension's persistent background page to take some action (either a local/remote server through WebSockets, or locally via the Native Messaging Host API). User-driven? Check.
• User creates a set of rules on www.PingMeWhenPS3sAreInStock.com, in order to know when PS3s are in stock at MSRP, to avoid paying scalpers exorbitant rates. These rules cause alerts to be pushed down at arbitrary times to a browser extension's persistent background page as indicated, driven through a real-time WebSocket subscription. User-driven? Check. (The system is carrying out the user's intended actions, albeit asynchronously.)
• User uses an iPhone/Android app as a scanner for a checkout/item-lookup system. This system's intended flow is for the phone to scan an item, and for the browser extension to then open a new tab containing information about that item. The user expects this to work immediately upon scanning, without needing to take any additional tedious action within the browser. This is accomplished via a WebSocket listener running in a persistent background page. User-driven? Check.

In each of these scenarios, it's easy to tell from a user-centric perspective that the user was responsible for the action.

In the modern web, many browser extensions (and many websites!) exist as a part of a wider system. A user operates other components of that greater system, and expects direct and immediate impacts upon other parts of that system (including, but not limited to, browser extensions).

Only from a browser-centric perspective could these example flows not be perceived as "user-driven".

To recap:

Service workers are user-hostile, as they operate upon the flawed assumption that the user doesn't exist outside of a web browser's frontend. They view the user/browser interaction from the browser's perspective, rather than from the user's.

[ameshkov]

We see benefits in SW that they are a standardized execution environment with defined lifetimes by design

It is indeed defined when we're speaking about websites, but I've seen no definition of the service worker's lifetime in regards to the browser extension besides this single phrase:

First, service workers are terminated when not in use and restarted when needed (similar to event pages).

The spec does not define what "in use" means.

For instance, if there's an active web page the extension has access to, does it mean that the service worker is in use?

[nodeticswww]

Manifest v3 is a disaster for our WebExtensions: Feedbro (RSS Feed reader) and PageProbe (page monitoring app).

Both WebExtensions need a persistent background page that is used for scanning external resources (RSS feeds + pages) and for fast look-ups an in-memory cache which is not quick enough to be implemented with storage APIs. Also audio notifications become impossible (desktop notifications is a question mark?).

Due to the following limitations it will be impossible to implement Feedbro & PageProbe using manifest model v3 so all our work is lost due to this change. Feedbro for example has around 150 000 daily active users (all browsers combined).

Hasn't anyone considered WebExtensions that need to do frequent background processing, fetching of data from external resources and need a fast in-memory lookup cache? Seems like a massive oversight and cripples the power of WebExtensions to a laughable level.

1. background page is replaced with short lived service workers
2. webRequest API is no longer available and is replaced with declarativeNetRequest
3. DOMParser is not available in a service worker (hard to do processing of loaded external pages etc)
4. Audio cannot be played in a service worker (good bye user audio notifications)
5. XMLHttpRequest is not available in a service worker
6. Timed events cannot be flexibly managed in a service worker (alarms is not good enough)

[gdh1995]

Service Worker, used by Manifest V3 is also harming VIM-like keyboard shortcut extensions, including Vimium (400,000+ users on Chrome Web Store) and my "Vimium C".

Such extensions allow users to write a long list of mapping some keys to some commands with arbitrary options. And some of them even allow users to add arbitrary JS functions as commands. Then it can be a very heavy task to parsing the list.

Vimium and some others provide its Vomnibar as a dialog iframe, to let users search in history, bookmarks, tabs, and it even show google suggestions. This means they will keep all browsing history in memory and tracing history changes manually - which works well in Manifest V2, while in Manifest V3 it may need to request history again and again. I can't imagine how much more CPU it will cost.

Some other disadvantages, like lacking clipboard, seems "tolerable", while their working around / replacement using content scripts may cause new bugs and can't be reproduced easily, because of various web page contexts.

BTW, for those network request blockers, restarting a service worker and reparsing tons of rules will apparently costs more CPU and energy. I believe Google completely ignored their existence on purpose, in order to earn much more from AD business.

[sondreb]

As there is less than a year left for a huge amount of popular extensions to migrate, many which cannot migrate due to the limitations of V3, greater warnings and alerts should be distributed to ensure that both developers and users know that migration must happen or extensions will stop working, in less than a year.