Allow SDK usage from service worker

[bwindels]

The service worker is growing in size, so everything the browser polls for an update, it downloads the whole file. Also, the code is growing in complexity, and it is becoming hard to navigate the single large file. All the replaced variables should be defined in the file, along with an import, and the bundle with the real code should go in a file that has a content hash on it.

We want to keep the ability to run the service worker without transpilation in both latest FF and Chromium, so we need to use importScripts (can't do this anyway if we want to run sync from service worker).

We will end up using multiple files in the main import, and we would want those bundled up, so we would need to build a bundle for the service worker code that is worker-compatible (e.g. exports it symbols through a global variable) and then with the build script replace the name we do importScripts of in the service worker.

• service-worker/
  • service-worker.js
  • main.js
  • push.js
  • cache.js

being bundled up into sw.js and sw-bundle-34985934.js. Most of the work will be in the build.mjs script, the rest in reorganizing the service worker code into multiple files.

The olm worker currently isn't very big but might eventually also benefit from this.

[bwindels]

How could this tie in with running sync from the service worker (separate issue, not part of this)? The only way I can think off would be to:

• not use ES modules anymore in production
• have a bundle for the sdk and a bundle for the rest, that set a global variable
• the page loads them with script tags
• the service worker importScripts the sdk, and then finds it at the same global variable.
• This means again the service worker can't be run locally for development ... so perhaps we should just bite the bullet and say you can only run the service worker locally in a browser that supports imports in a worker (e.g. chrome at the time of writing). We'd have to transpile the service worker code to use importScripts to load its bundle though.

[bwindels]

So, we'll need a rollup plugin where we can define files where imports should be replaced with importScripts (in our case sw.js). We'll also set it up that we have a main.js for the sw, and we turn that into a bundle. so we'll have:

//... service worker constants...
import {startServiceWorker} from "./main.js";
startServiceWorker();

which will be turned into

//... service worker constants...

// defines __sw_main as global variable
importScripts("sw-main-13924839.js");
const {startServiceWorker} = __sw_main;
startServiceWorker();

so we'd need to look for import declarations in a transform hook, and transform them to a function call of importScripts.

We'll also need to use manualChunks to set up these chunks:

• sdk (anything in matrix/ ... and anything imported from files therein assuming we did https://github.com/vector-im/hydrogen-web/issues/310?)
  • what about the platform code though? it would mean it can only become an import if the API is also available in a worker...
  • perhaps we should have a different platform (sub?)class for the sdk and for view models? matrix uses these platform apis atm:
    • clock (needed, same as non-worker)
    • sessionInfoStorage (needed, same as non-worker)
    • random (needed, same as non-worker)
    • logger (needed, same as non-worker)
    • storageFactory (needed, same as non-worker)
    • createBlob (needed, same as non-worker)
    • request (needed, fetch, same as non-worker)
    • notificationService (needed for #312, same as non-worker)
    • crypto (turn into import)
    • encoding (turn into import)
    • version (turn into import)
  • even though platform api is the same, the initialization might be different (e.g. service worker needs its own logging db), so we need a ServiceWorkerPlatform and a DOMPlatform as SDK platforms, and a subclass of DOMPlatform (ViewDOMPlatform) with extensions for what the view models need (saveFileAs, ...).
• sw-main (anything from sw main.js file that is not sdk and not sw.js itself)
• app-main (anything from app main.js file that is not sdk)

[bwindels]

how about code used in sdk like the observables we (might) use in app-main as well? separate chunk?

[bwindels]

also see https://github.com/surma/rollup-plugin-off-main-thread

seems like importScripts can work with AMD modules? Could we compile the common code to AMD and also use that for loading the SDK from the document/main thread?

But if not loading our main bundle with a <script type="module"> tag, then how do we determine whether we should load a "modern" (vs legacy) bundle? I guess can programatically determine module support and load the modern AMD bundle if that is the case. We don't actually use the module support but rather use it as a baseline for our modern browser support.

[bwindels]

Note that vite also has support for compiling web workers, but that just throws everything in an IIFE. So it won't help us with this issue really, as going that way would make the service worker huge, which is problematic as it can't be cached.