search

WebAudio / web-midi-api

The Web MIDI API, developed by the W3C Audio WG
http://webaudio.github.io/web-midi-api/
Other
321 stars 55 forks source link

Make MIDIAccess transferable and exposed to workers #256

Closed mjwilson-google closed 1 day ago

mjwilson-google commented 8 months ago

This is to fix #99.

Some things to note:

cwilso commented 8 months ago

I think at this point it's more useful in Worklet (so it's exposed in AudioWorklet) than worker.

padenot commented 8 months ago

I think at this point it's more useful in Worklet (so it's exposed in AudioWorklet) than worker.

We've discussed this at TPAC and concluded this is best done by authors, like it is in native code.

padenot commented 8 months ago

I'm wondering if we should make it transfer-only, and refuse regular cloning. I think it would simplify things somewhat.

hoch commented 8 months ago

I think at this point it's more useful in Worklet (so it's exposed in AudioWorklet) than worker.

Re: @cwilso - What @padenot stated above means that the group believes this works better with Worker and SAB. Also the feature set in the WorkletGlobalScope is intentionally minimum, so we want to make sure all interfaces used by Web MIDI are properly supported by it.

If incoming MIDI events are bound to AWGS, we'll see a lot of bursty actions in the performance of audio callback. Creating new events is "allocation", and we don't want to do that on the RT audio thread.

I'm wondering if we should make it transfer-only, and refuse regular cloning. I think it would simplify things somewhat.

@padenot Do you have any example of that?

@mjwilson-google Have we confirmed that all the interfaces in the Web MIDI API are supported in the WorkerGlobalScope? I am assuming it does, but it doesn't hurt to check. :)

cwilso commented 8 months ago

@hoch hmm, yeah, you're right, makes sense. I retract my previous statement. :)

mjwilson-google commented 8 months ago

@hoch I don't know how to confirm that the interfaces are supported in WorkerGlobalScope. How do we do that?

hoch commented 8 months ago

From the spec(https://webaudio.github.io/web-midi-api), I see these interfaces:

5.1 MIDIInputMap Interface 5.2 MIDIOutputMap Interface 5.3 MIDIAccess Interface 5.4 MIDIPort Interface 5.4.1 MIDIInput Interface 5.4.2 MIDIOutput Interface 5.5 MIDIMessageEvent Interface 5.6MIDIConnectionEvent Interface

This PR only has a change for MIDIAccess, so it needs to be expanded to add the Worker exposure (e.g. [Exposed=Worker]) to all the interfaces.

mjwilson-google commented 8 months ago

This PR only has a change for MIDIAccess, so it needs to be expanded to add the Worker exposure (e.g. [Exposed=Worker]) to all the interfaces.

Done. Is it fine to keep Transferable only on the MIDIAccess interface?

padenot commented 8 months ago

Done. Is it fine to keep Transferable only on the MIDIAccess interface?

In terms of developer ergonomics, I think it's fine.

The author performs the request from the main thread. This deals with the all the permission business. The author then transfers the MIDIAccess object, and does all the midi things in the Worker. This means that there is generally only one thread that deals with MIDI (it's always possible to get other MIDIAccess however).

We can also decide to allow sending ports and such to the worker, but I think I like that style less.

mjwilson-google commented 8 months ago

@padenot How does the current state look? Do you want to explore making the object transfer-only? Would the way to do that be to mark it as non-serializable, to disable cloning?

We will need a follow-up PR to further define the behavior when transferred before closing the linked issue.

svgeesus commented 3 days ago

Should this be merged? We have an approval from @hoch and a "looks okay" from @padenot

mjwilson-google commented 3 days ago

Yes it should be merged. I have been waiting to update the working draft first. Sorry about the long delay.

svgeesus commented 2 days ago

Will merging trigger the automatic re-publishing, or is that not set up yet?

I really want to see a WD that is more recent than 2015 :)

mjwilson-google commented 2 days ago

Will merging trigger the automatic re-publishing, or is that not set up yet?

I really want to see a WD that is more recent than 2015 :)

I just put up pull request #262 to add the auto-publish workflow. I think I need your help for the Echinda token; could you please take a look?

Once that is merged, then yes merging this should automatically update the Working Draft and we can start merging other changes more quickly again (which will also update the Working Draft).

Thank you for checking in on this, and sorry again for the long delay.

svgeesus commented 1 day ago

So I am seeing the new ED https://webaudio.github.io/web-midi-api/ W3C Editor's Draft 04 July 2024

But https://github.com/WebAudio/web-midi-api/actions/runs/9794115046/job/27043463563 says TR publication was "skipped". And nothing at https://lists.w3.org/Archives/Public/public-tr-notifications/2024Jul/

I don't understand why.