[v2] ServiceWorker cannot be registered from wails:// scheme

[micahyoung]

Description

For a built app, using the wails:// protocol for assets, a Service Workers cannot be registered, ex: for a call like:

navigator.serviceWorker.register("./serviceWorker.js")

Debug console result:

Unhandled Promise Rejection: TypeError: serviceWorker.register() must be called with a script URL whose protocol is either HTTP or HTTPS

This is likely due to browsers only allowing http: and https: for service worker URLs, and not wails:. This unfortunately appears to be part of the serviceWorker.register() spec:

6. If scriptURL’s scheme is not one of "http" and "https", reject p with a TypeError and abort these steps.

To Reproduce

1. Init vanilla app

   wails init -n my-serviceworker

2. Add an empty file that will be used as the service worker:

   touch my-serviceworker/frontend/serviceWorker.js

3. Run with dev

   wails dev

4. Right click on window -> Click Inspect Element -> Click Console and run a call that will succeed:

   await navigator.serviceWorker.register("./serviceWorker.js")
   // result
   ServiceWorkerRegistration {installing: null, waiting: ServiceWorker, active: ServiceWorker, scope: "http://localhost:34115/", updateViaCache: "imports", …}

5. Build and run (with -debug):

   wails build -debug
   
   my-serviceworker/build/bin/my-serviceworker.app/Contents/MacOS/my-serviceworker

6. Right click on window -> Click Inspect Element -> Click Console, run the same code and see the following error message:

   await navigator.serviceWorker.register("./serviceWorker.js")
   // result:
   TypeError: serviceWorker.register() must be called with a script URL whose protocol is either HTTP or HTTPS

Expected behaviour

Service workers work with build as they do with dev and with a normal web server

Screenshots

No response

Attempted Fixes

• Add serviceWorker.register to my-serviceworker/src/main.js instead of just in the Console
  • Result: fails with same error
• Using an absolute path, full URL, etc (ex: /serviceWorker.js, wails://wails/serviceWorker.js)
  • Result: fails with same error
• Inlining a service worker using data: protocol, as suggested on Stack Overflow
  • Result: fails with same error

• Make serviceWorker.js a module and use import.meta.url in a wrapper function like:

  // my-serviceworker/src/serviceWorker.js
  export default function registerServiceWorker() {
      navigator.serviceWorker.register(import.meta.url)
  }
  
  // my-serviceworker/src/main.js
  import registerServiceWorker from `./serviceWorker.js` 
  registerServiceWorker()
  ...

  • Result: fails with same error

System Details

Wails CLI v2.0.0-beta.36

Scanning system - Please wait (this may take a long time)...Done.

System
------
OS:     MacOS
Version:    12.2.1
ID:     199506
Go Version: go1.18.1
Platform:   darwin
Architecture:   arm64

Wails
------
Version:    v2.0.0-beta.36

Dependency          Package Name    Status      Version
----------          ------------    ------      -------
xcode command line tools    N/A     Installed   2392
npm                 N/A     Installed   8.5.5
*upx                N/A     Installed   upx 3.96
*nsis               N/A     Available   

* - Optional Dependency

Diagnosis
---------
Your system is ready for Wails development!
Optional package(s) installation details: 
  - nsis : Available at https://nsis.sourceforge.io/Download

Additional context

No response

[micahyoung]

It's probably worth mentioning that Chrome Extensions also don't support serviceWorker.register, instead service workers are registered via the manifest. I'm not sure why it does that but it could be for the same reason (their scheme is chrome-extension)

I feel like perhaps a "frontend:service_worker": "" option to wails.json (or something similar) could be acceptable, from a user's perspective. However, I would guess there would be a fair bit of internal work required to either make an alternative service worker implementation, or otherwise bypass the scheme check.

[micahyoung]

Electron appears to expose an API function for declaring a scheme as "secure", thereby allowing Service Workers on that protocol.

protocol.registerSchemesAsPrivileged([
  { scheme: 'foo', privileges: { bypassCSP: true } }
])

I'm not very familiar with C++ (nor Wails internals) but I think the Electron source appears to call several underlying WebKit APIs to setup the scheme, that perhaps Wails could use as well.

https://github.com/electron/electron/blob/95a9ff952c231eca2f04e7f444b2e1818b4f65a0/shell/browser/api/electron_api_protocol.cc#L100-L162

[leaanthony]

Thanks for taking the time to look at this! There was an existing issue talking about service workers so I am aware it's important. If there is a particular platform you're interested in getting this done for, I'm happy for you to start on that and add it as a platform specific option. Then we can roll it out to the general config as we add the other platforms. We can tag the feature as experimental.

The quickest way to get this in is doing what you're doing: finding out how it works and what might be needed. Once we crack that nut, I'm happy to invest the time to get it over the line 👍

[micahyoung]

Thanks for the guidance (and the great framework!). I'm happy to keep digging into this one as it's starting to feel more comprehensible as I go along.

My platform of primary interest is MacOS (though Windows would be close after) but after digging through this codebase more, I see WKWebview is used to register the scheme:

https://github.com/wailsapp/wails/blob/22d3f93b5205f32118fef11170203d85a6261fa7/v2/internal/frontend/desktop/darwin/WailsContext.m#L212-L215

After looking through the WKWebview API, I don't (yet) see way to set scheme security though. The docs for setURLSchemeHandler don't give any suggestions. I'll keep digging though, first to hopefully find the proper WKWebview API for setting it, otherwise my next thought is to see if there's a way to bypass WKWebview and call to underlying Webkit API directly. Any thoughts on whether calling Webkit directly could be possible?

[stffabi]

@micahyoung thanks for looking into ServiceWorker support.

There was #1210 for support on Windows, where we changed the scheme from file:// to http(s):// to support them. So Windows should be already supporting them. Thanks to your link to the Electron API I've found the WebKitSecurityManager, which hopefully would allow to register the scheme as secure and to bring support for ServiceWorker on Linux. Unfortunately I was also unable to find the corresponding API on WKWebView on MacOS, I suspect Apple doesn't make it publicly available. There's also not much of information for ServicesWorkers on WKWebView out there apart from "they are supporter on app bound domains on iOS". Maybe we could try to contact the Apple Support with a TSI?

[micahyoung]

Thanks for the additional context, that solution for Windows from #1210 makes a lot of sense and I'll try that in my MacOS fork. If it works, maybe that could be cross-platform, opt-in config option or something.

Unfortunately I was also unable to find the corresponding API on WKWebView on MacOS, I suspect Apple doesn't make it publicly available. There's also not much of information for ServicesWorkers on WKWebView out there apart from "they are supporter on app bound domains on iOS".

This lines up with what I was seeing, I did spot a private WKWebView function but I couldn't find a way to trigger it, nor could I spot anything public that seems to wrap it. But I'm certainly no MacOS expert though so someone else might have better ideas: WKProcessPoolPrivate.h:L89:

- (void)_registerURLSchemeAsSecure:(NSString *)scheme WK_API_AVAILABLE(macos(12.0), ios(15.0)); 

As for the app-bound domains approach, I tried adding wails (also tried wails://wails and others) to the Info.plist with an WKAppBoundDomains array and that also didn't make any difference.

I am starting to get the impression though WKWebView may only provide narrow functionality for custom schemes by design, since https (the only "secure" scheme) appears to be very locked-down itself. Perhaps opening a Apple Support ticket could help but I'm not feeling too optimistic.

[micahyoung]

One more thread I'm looking at is playing around with Content-Security-Policy HTTP Headers, to potentially mark wails:// as secure. The docs have some interesting hints:

worker-src
    Specifies valid sources for Worker, SharedWorker, and ServiceWorker scripts.
...
upgrade-insecure-requests
    Instructs user agents to treat all of a site's insecure URLs (those served over HTTP) as though they have been replaced with secure URLs (those served over HTTPS). This directive is intended for websites with large numbers of insecure legacy URLs that need to be rewritten.

Sounds potentially promising, but I don't have much experience with CSP headers though, so I might be missing something there. I'll post anything that comes out of it.

Additional note: this appears to be the relevant bit of WebKit source that checks the scheme (there are other surrounding checks, but this block emits the exact error I'm seeing):

ServiceWorkerContainer.cpp:171

if (!jobData.scriptURL.protocolIsInHTTPFamily() && !jobData.isFromServiceWorkerPage) {
    CONTAINER_RELEASE_LOG_ERROR("addRegistration: Invalid scriptURL scheme is not HTTP or HTTPS");
    promise->reject(Exception { TypeError, "serviceWorker.register() must be called with a script URL whose protocol is either HTTP or HTTPS"_s });
    return;
}

[stffabi]

Thanks for the additional context, that solution for Windows from https://github.com/wailsapp/wails/issues/1210 makes a lot of sense and I'll try that in my MacOS fork. If it works, maybe that could be cross-platform, opt-in config option or something.

Using "http://" (previously "file://") instead of the custom "wails://" on Windows was/is more of a workaround because at the moment Webview2 on Windows doesn't support registering custom schemes, but it allows intercepting all requests so it was possible to use "http://". AFAIK there's no api to intercept calls on WKWebView on MacOS without registering a custom scheme and registering "http"/"https" seems not be possible. So I have doubt we will get this supported on MacOS.

[micahyoung]

Ah, that makes sense @stffabi . Then to me, it feels like fixing this on MacOS right now is mostly infeasible then, as the guard (scriptURL.protocolIsInHTTPFamily) will only allow service workers from http/https requests, which we can't intercept on MacOS.

Now very interestingly, the commit history on that exact guard line shows a potential solution:

WebKit@cf08141f - Oct 4, 2021

... Add new [WKWebView loadServiceWorker:(NSURL *)] to load a service worker in a web view. When called, it causes us to create a page with HTML start registers the service worker with the provided URL. ...

This seems to be something built for our use-case, which we could use to bypass the guard on serviceWorker.register and load as ServiceWorker directly. Unfortunately, it was only added a few months ago, and it's not in my local SDK (12.3) as far a I can tell, but that feels probably correct way-forward for MacOS.

[stffabi]

Oh that sounds promising, maybe it gets included in the next major MacOS version.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[leaanthony]

@micahyoung - Did you try with later versions of macOS? Wondering if this is now resolved.