WebAudio / web-audio-api

The Web Audio API v1.0, developed by the W3C Audio WG
https://webaudio.github.io/web-audio-api/
Other
1.05k stars 166 forks source link

Event Loop execution in `closed` state #2568

Closed daxpedda closed 7 months ago

daxpedda commented 7 months ago

Describe the feature Clarification on event loop execution when the state has reached closed.

Describe the feature in more detail Currently the closed state is defined as:

This context has been released, and can no longer be used to process audio. All system audio resources have been released.

This however doesn't talk about the event loop still running or not. This matters particularly to figure out if the worklet can still run user-code. The use-case here comes from Wasm: it needs to know when it is safe to clean up threads. If Wasm is called into after the thread is cleaned up, it could lead to undefined behavior.

Dedicated workers can call Worker.terminate(), which makes sure the worker can't execute user-code anymore. But audio worklets have no real equivalent if the closed state still allows user-code to be run.

I don't know enough about the specs in question to know if stopping the event loop would also stop Promises, which could potentially still run user-code even after the event loop is stopped.

FWIW: this could be worked around by making sure that after cleaning up the thread all references to Wasm (e.g. exported functions) are set to undefined. This however is not always possible, as users are able to dynamically create references to the Wasm module (e.g. ArrayBuffer views) which can't easily be controlled by tools or libraries.

This has been originally raised at Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1878516. Chrome currently (AFAIK) doesn't run user-code after the closed state is reached.

karlt commented 7 months ago

The close() method will not automatically release all AudioContext-created objects.

A close() is also available on the MessagePort and useful for releasing unreferenced resources, but script can still be triggered on the worklet thread through BaseAudioContext.audioWorklet.addModule().

Making AudioContext.close() also close MessagePorts could be a breaking change for any clients that, for example, might collect statistics after AudioContext.close(). If Chrome has always done this, however, then perhaps that might not be a problem. I wonder what Chrome does to the MessagePorts and what it does when addModule() is called. I think I'd prefer something explicit over having messages queue but never run because the event loop stops processing them.

The worklet's lifetime is explicitly terminated only when "destroying the Document". Worklet has some provision for user agents choosing to terminate, but AudioWorkletGlobalScope forbids: AudioWorkletGlobalScopes must not be terminated arbitrarily by the user agent. Worklets may of course be garbage collected if content scripts no longer have access or cannot be triggered.

If the AudioWorkletGlobalScope were terminated, addModule would create another. This would have no references to the previous Wasm module, unless the previously added modules somehow can obtain a reference again.

I'm not familiar enough with Wasm to fully understand the requirements and when this becomes an issue. Would similar issues apply to other Worklet types or is this problem specific to AudioWorklet because it allows communication through MessagePort?

I'm not clear how Worker.terminate() would be useful for this purpose, given that it runs in parallel and I don't see any notification of completion. I'm not familiar with the history or purpose of this method.

daxpedda commented 7 months ago

Would similar issues apply to other Worklet types or is this problem specific to AudioWorklet because it allows communication through MessagePort?

I'm not familiar enough with other worklet types, but I think it would indeed apply to any worklet type.

I'm not clear how Worker.terminate() would be useful for this purpose, given that it runs in parallel and I don't see any notification of completion. I'm not familiar with the history or purpose of this method.

Indeed Worker.terminate() is not immediate and there is no notification of completion AFAIK. But workers can be prevented from running anything by just blocking indefinitely with Atomics.wait(), which can't be done with worklets as they don't support blocking.

The worklet's lifetime is explicitly terminated only when "destroying the Document". Worklet has some provision for user agents choosing to terminate, but AudioWorkletGlobalScope forbids: AudioWorkletGlobalScopes must not be terminated arbitrarily by the user agent.

I think this resolves the clarification issue! Thank you for finding it for me.


Currently audio worklets are not garbage collected in any browser (Chrome, Firefox, Safari) when a sufficiently complex Wasm module is instantiated in them, even after dropping all references to JS values. I assume it is difficult to determine if the Wasm module could execute any code (e.g. Atomics.waitAsync() could be used to wake up and run code at any time, or a FinalizationRegistry could be used to call into the module when something is garbage collected).

I confirmed in Chrome and Firefox that without a Wasm module audio worklets are successfully garbage collected and it can be detected by using FinalizationRegistry.

In conclusion the closed state can't be used to solve this problem.

hoch commented 7 months ago

Thanks both for the discussion! FWIW, I agree with @karlt's response.