search

whatwg / html

HTML Standard
https://html.spec.whatwg.org/multipage/
Other
8.13k stars 2.67k forks source link

Worklet agents and the event loop #10136

Open annevk opened 8 months ago

annevk commented 8 months ago

As @mhofman discovered in https://github.com/tc39/proposal-shadowrealm/issues/398#issuecomment-1940380165 there's something broken about the current setup. Agents are assumed to be 1:1 with an event loop, but we don't define an event loop for worklets (as we don't want them to have one). However, this means that certain features such as FinalizationRegistry effectively crash (see HostEnqueueFinalizationRegistryCleanupJob(finalizationRegistry)).

I think this means we either need to have non-event-loop agents that don't expose these features or we need to give worklets a minimal event loop. I think the latter is very much undesirable for good audio, but I'll defer to @padenot on that.

cc @syg @padenot @domenic @rniwa @smaug----

padenot commented 8 months ago

https://webaudio.github.io/web-audio-api/#rendering-loop has an attempt to make this work.

As far as the Web Audio API is concerned, it's not that it's not possible, it's that the event loop cannot execute traditionally, executing the tasks when there exist: instead, the tasks are processed near the beginning of a "render quantum" when the OS calls a callback on a real-time thread.

We might be able to make this clearer or more correct (if it's not), paging @hoch in any case.

hoch commented 8 months ago

A concept of event loop with real-time constraints would be a decent solution to this problem, but I think that's a rather significant change with so many uncertainties to introduce after a decade of the API launch. So without such system, the rendering loop algorithm that we have in Web Audio API is the best we can do.

+1 on making it more clearer or more correct.

annevk commented 8 months ago

I think the problem now is that there are certain JS APIs that assume there's an event loop (as defined in the HTML Standard), but that's not the case for worklets. So either those JS features cannot work in worklets or they work in some undefined manner.

It does seem like Web Audio attempts to integrate with this maybe through the "associated task queue", but it doesn't seem particularly sound.