atob() is missing in AudioWorkletGlobalScope

[juj]

In https://github.com/emscripten-core/emscripten/issues/19845#issuecomment-1660395345 a developer has observed that atob() is missing in AudioWorkletGlobalScope, even though the function is available in WorkletGlobalScope.

The function was expected to be there since AudioWorkletGlobalScope inherits from WorkletGlobalScope ([1], [2]).

Could the function be added?

[padenot]

A WorkletGlobalScope is unrelated to WorkerGlobalScope: https://html.spec.whatwg.org/multipage/worklets.html#worklets-global.

It intentionally has nothing in its interface, just what ECMAScript provides and of course the AudioWorkletGlobalScrope API. Since atob is exposed on Window and Worker, this behaviour is correct.

I'd like to understand the reason why atob is needed here, because it really shouldn't be. It hints at the fact that a long and blocking operation is happening in the AudioWorkletGlobalScope, and this shouldn't generally happen, so it tells me we're potentially missing a functionality, or documentation on how to do something in a way that is real-time safe (generally it boils down to using transferable and using SharedArrayBuffer).

Reading the thread you linked, it seems to be during initialization -- at this moment in the user's app, the audio isn't flowing, so we don't have real-time constraints. If that's the case, the preferable way to do this is to decode the data into a buffer (from the main thread or worker thread if it's large, and we don't want to block), and transfer the data using postMessage in a zero-copy way.

Let me know if we can help further.

[guest271314]

I'd like to understand the reason why atob is needed here, because it really shouldn't be.

Reading the issue

With 3.1.41 the output single js file had a function base64Decode

I have used this for encoding and decoding base64

// https://stackoverflow.com/a/62362724
function bytesArrToBase64(arr) {
  const abc =
    'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/'; // base64 alphabet
  const bin = (n) => n.toString(2).padStart(8, 0); // convert num to 8-bit binary string
  const l = arr.length;
  let result = '';

  for (let i = 0; i <= (l - 1) / 3; i++) {
    let c1 = i * 3 + 1 >= l; // case when "=" is on end
    let c2 = i * 3 + 2 >= l; // case when "=" is on end
    let chunk =
      bin(arr[3 * i]) +
      bin(c1 ? 0 : arr[3 * i + 1]) +
      bin(c2 ? 0 : arr[3 * i + 2]);
    let r = chunk
      .match(/.{1,6}/g)
      .map((x, j) =>
        j == 3 && c2 ? '=' : j == 2 && c1 ? '=' : abc[+('0b' + x)]
      );
    result += r.join('');
  }

  return result;
}
// https://stackoverflow.com/a/62364519
function base64ToBytesArr(str) {
  const abc = [
    ...'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/',
  ]; // base64 alphabet
  let result = [];

  for (let i = 0; i < str.length / 4; i++) {
    let chunk = [...str.slice(4 * i, 4 * i + 4)];
    let bin = chunk
      .map((x) => abc.indexOf(x).toString(2).padStart(6, 0))
      .join('');
    let bytes = bin.match(/.{1,8}/g).map((x) => +('0b' + x));
    result.push(
      ...bytes.slice(0, 3 - (str[4 * i + 2] == '=') - (str[4 * i + 3] == '='))
    );
  }
  return result;
}

[juj]

The intent with Wasm Audio Worklets is to provide Wasm developers a function callback entrypoint that they can implement in their same codebase as the rest of their C/C++ program.

To support this, we want to load the same shared .wasm Module and Memory in the Wasm Audio Worklet as the main thread and other Workers use.

This also implies that the same main .js file is loaded, since that defines the wasm imports and exports.

However that causes the issues, because the same built-in JS APIs are not available in the Audio Worklet. So code sharing becomes unfeasible, or requires manually reimplementing polyfills for all these APIs.

JS functions such as atob(), performance.now(), TextDecoder/Encoder and Math.*() are commonly used in the shared JS code, many only at the initial parsing state. atob() for example can be used to decode an embedded wasm data section.

I think the philosophy of ”providing nothing” in the audio global scope is conflicting with the important use case of sharing code with audio worklet and other JS threads.

Wouldn’t it be better to have consistency and parity in Audio Worklets with respect to the other JS computing contexts?

[chrisguttandin]

@juj Wouldn't it be better from an architectural standpoint to load the wasm code only once on the main thread and transfer it to the worklet and the workers instead of loading it on the main thread, the worklet, and the workers?

It's of course possible that I missed something obvious here.

[juj]

load the wasm code only once on the main thread and transfer it to the worklet and the workers

That is what Emscripten does. Wasm Module is passed to Audio Worklet and Audio Worklet instantiates the Module (or in MINIMAL_RUNTIME build mode in a more optimized fashion that reuses more JS code between audio worklet and main thread/worker thread JS)

But because of the issue that Audio Worklets don't get most of the same global APIs, this JS code sharing results in ifdeffing and gating to guard against Audio Worklet scope missing things. In addition to atob(), some additional things that I could find with a short search:

• AudioWorklet does not have performance.timeOrigin
• AudioWorklet does not have performance.now
• AudioWorklet does not have postMessage()
• AudioWorklet does not have TextDecoder

[chrisguttandin]

@juj thanks for your response. Is Emscripten using these APIs at runtime? I always thought they are only used when preparing the binary response during compilation. That's why I thought AudioWorkletGlobalScope doesn't need them since it only deals with instantiating already compiled modules anyway. Does Emscripten for example support to call performance.now() from within the WASM code?

[juj]

I always thought they are only used when preparing the binary response during compilation.

They could be called either at global JS scope of the reused JS file, or as a JS library function that is called from Wasm.

Does Emscripten for example support to call performance.now() from within the WASM code?

Yes, it does. See AudioWorklet does not have performance.now.

[padenot]

Related: https://github.com/WebAudio/web-audio-api/issues/2499