WebAudio / web-midi-api

The Web MIDI API, developed by the W3C Audio WG
http://webaudio.github.io/web-midi-api/
Other
326 stars 49 forks source link

midi clock support? #232

Open Pomax opened 2 years ago

Pomax commented 2 years ago

Are there any plans to also make sure there's MIDI clock support, in the sense that the browser has a mechanism for starting a stable clock with event emission every 24th of a quarter note, based on a BPM value (or even mirroring the MIDI spec, where the tempo is set by specifying the number of microseconds per quarter note).

Right now our options are pretty crazy bad, and having a reliable time keeper on the JS side would enable a bunch of work that's currently just impossible (both for syncing MIDI devices using the browser as authority, as well as--in good old JS tradition-- complete spec-unrelated work that just needs an actually reliable interval trigger).

djipco commented 2 years ago

While it's not a metronome per-se, did you check out Chris Wilson's approach to timing?

Having said that, it would be awesome to have a reliable and precise timing mechanism capable of dispatching events. I'm just not sure if and how JavaScript's architecture could permit that.

Pomax commented 2 years ago

Indeed, it's also referenced by the link from my comment, and I'm literally working on a fork of his code to make it more 2022 JSish, but... it's way more code than anyone should ever have to write to get a reliable (midi) clock going, and it's still not properly accurate (there's stable lag anywhere from 20ms to over 100ms depending which device you're running on, and drift between ticks that ranges from negligible to 10ms thanks to a combination of setInterval and postMessage, which is very not negligible).

Especially for Web audio and MIDI, we still need a proper high precision trigger timer (like how we have performance.now() for when Date.now() isn't accurate enough, although of course firefox has helpfully made them the same resolution, so that's not great). Code that only works on a tower desktop that happens to have enough free resources for everything to run as fast as possible is "we're not there yet" code =)

So, as the MIDI spec has the concept of the clock built right into it, it would make sense to have some kind of MIDIClock or MIDI.Clock etc. global that can be told to create timer instances with some BPM or micros-per-quarter value, that generates midi tick messages (explicitly not using the current JS timer definition that back setTimeout/setInterval) that can be forwarded to a MIDI device, and/or have code kick in when ticks fly by.

boourns commented 2 years ago

So, as the MIDI spec has the concept of the clock built right into it, it would make sense to have some kind of MIDIClock or MIDI.Clock etc. global that can be told to create timer instances with some BPM or micros-per-quarter value, that generates midi tick messages (explicitly not using the current JS timer definition that back setTimeout/setInterval) that can be forwarded to a MIDI device, and/or have code kick in when ticks fly by.

While your proposal may solve the issue of generating a stable MIDI clock in isolation, it would not integrate well with an audio application that is trying to syncronize MIDI clocks, MIDI events, and audio input or output.

I would like to propose an alternative solution: please add direct access to the Web MIDI API from the audio thread.

if the Web MIDI API was available on the audio thread, then we could do low-latency scheduling of all MIDI events in a sample-accurate manner by writing an AudioWorklet that emits MIDI during it's process function. With this API somebody could build a MIDI.Clock AudioNode / AudioProcessor pair that implements the API described in the above comment.

I'm still getting up to speed with the history of web midi spec and if the audio thread has already been proposed or discussed i apologize.

I've written a few sample-accurate MIDI sequencers with this approach built as Web Audio Modules, for example this piano roll but the problem I have is there is no way to emit the MIDI messages to a hardware MIDI port without first posting to the main thread, introducing all sorts of timing issues.

image
djipco commented 2 years ago

I would like to propose an alternative solution: please add direct access to the Web MIDI API from the audio thread.

This is definitely worth looking into. Another suggested approach is to make the Web MIDI API avaible in a worker thread.

cwilso commented 2 years ago

@djipco Web MIDI availability in workers is issue #99.

mjwilson-google commented 1 year ago

I will tentatively schedule this for CR, following #99, but I would like to discuss how Web MIDI and Web Audio interact with the Audio Working Group.

I think it would make it easier for application developers if we can schedule MIDI event timing from the audio render thread (this is how JACK works, for instance), but I still don' t have a good idea of what spec work would be involved and how much implementers would have to change to make that possible.

mjwilson-google commented 1 year ago

Audio Working Group 2023-10-05 meeting conclusions:

mjwilson-google commented 2 months ago

TPAC 2024 notes:

The spec says that this should be available from workers now, but no known implementation has implemented this change yet. Once we have it implemented we can see if that satisfies the use cases or if we need to change the spec.