Add MIDI support for program change messages

[totalgee]

A MIDI device I'm using (Dirtywave M8) has only 8 mono tracks, but you can switch "instruments" per note event, using MIDI program change messages. This even works if you switch immediately before (at the "same time" as a noteOn event). A use for this would be to play bass drum and snare drum (two different instruments) on the same MIDI channel, switching between them as needed. Another use case for program change would be to change instruments from time to time, e.g. every N bars, or for different sections of a performance.

Implementation note: if you have a noteOn and a programChange event at the "same time", the program change MIDI message should be sent first.

Maybe it could look something like this, assuming program 100 was a bass drum instrument and 110 was a snare:

new midi default time(1/4) prog([100 110]) note(0 0)  // or midiprog() ?

Note that, although it's difficult/impossible (!) to find using the webMidi API search feature, program change is supported in both version 2.5 and 3.0.

Version 3: https://webmidijs.org/api/classes/OutputChannel#sendProgramChange Version 2.5.3: https://webmidijs.org/archives/api/v2/classes/Output.html#method_sendProgramChange

[tmhglnd]

I will have a look to this also! I think prog([100 110]) makes sense. Other function names that could be possible: change(), program(), programChange(), and maybe a shortcut: pgm() or pc() (like there is also cc() )

I'm not sure what is the most intuitive name if you would want this function and just type it, my guess would just be programChange(), but it's long so I could add a few aliases for the function, that is not a problem.

[totalgee]

Sounds perfect, thanks. Yes, I see you tend to make aliases, which is good. I think my preferences would be program, prog or programChange, but I don't really mind.

[tmhglnd]

I added the program() method. Documentation here: https://tmhglnd.github.io/mercury/02-instrument.html#program

Give it a try to see if this does the trick! I tested it only locally on my computer (by sending program changes to general midi) and that seems to work. The change should be send before the note is played.

[totalgee]

Thanks! I did a quick test of it, and found that it seems to work (I've not tried yet on my main use case of the M8, which can switch "instantly" between programs). I did try on a VST instrument, and because it takes some small time to switch programs, the sound is cut off when using this feature...this is expected, it depends on each MIDI synth how it responds to program changes and how long it takes to complete the change.

However, one thing that might be a good idea is to only send the program change once (the first time after evaluation) in the case where you only specify a single value, not a list, like this:

new midi default time(1/4) program(33) note(0 0)

In this case, maybe you could just sent program 33 on the first run through, not with each new note...? I know this is different from other properties, though...

(I will test with the M8 later.)

[totalgee]

Just testing with the VST host program "Cantabile" and its MIDI Monitor, it appears (if it can be relied on) that the noteOn message is actually coming before the program change message (which would explain why the note is sometimes cut off when playing in some VSTs). The Mercury code was this:

set tempo 120
new midi default time(1/4) program([100 110]) note(0 0) length(0.99/4)

And the MIDI monitor shows this:

(Also, you'll note that the program number appears off-by-one, but it's fine, since MIDI messages are 0-based (0-127) but they are normally shown as 1-based (1-128). So while we put program 100 in Mercury (or SuperCollider), it appears as 101 in the MIDI monitor (and in the VST instrument).

[totalgee]

Also, if you're wondering why I put length(0.99/4), it's related to issue #43; if I put length(1/4) then the noteOn message arrives before the first noteOff message, as you can see here:

(first MIDI output for the following Mercury code)

set tempo 120
new midi default time(1/4) program([100 110]) note(0 0) length(1/4)

[totalgee]

Okay, I tested it with the M8, and it does work reasonably well. In Firefox (on Windows) I was able to get pretty reliable program switching on these "simultaneous" MIDI events (program change + noteOn), up to about 1/16th notes (switching every note) and a bit more than 120 bpm. It does fail to switch from time to time, the message probably arrives a bit later in some cases.

Interestingly, it was much more reliable (at that speed) on Firefox than Edge (which is Chromium-based). Under Edge it was missing lots of the events.

I wonder if it would help if you scheduled the "simultaneous" program change messages a tiny bit earlier, like 1ms, if it would help? So they're not exactly simultaneous...

Anyhow, thanks again for adding this so quickly, it is definitely a proof of concept and works in some cases (and can maybe be improved still).

[tmhglnd]

Thanks for testing! I made some small updates based on your feedback in this commit: 21db26de7324da0992abec6188c58d4a3b7b5bdd

1. Program change message is send 1ms earlier than the note
2. I subtracted 1ms from the duration that hopefully makes sure that the note-off is send before the next note on in the case you have time(1/8) length(1/8)

I'm not sure if 1ms is enough tho? But give it a try

[tmhglnd]

However, one thing that might be a good idea is to only send the program change once (the first time after evaluation) in the case where you only specify a single value, not a list, like this: In this case, maybe you could just sent program 33 on the first run through, not with each new note...? I know this is different from other properties, though...

It would be possible to implement this yes, but it does indeed go against how everything else works. Do you think some systems might have a hard time if they keep receiving the program change message even tho it doesn't change? I'm not sure if most MIDI devices already filter for this or if they try to change anyways even tho the number is the same. If that's the case then this could be a clear reason to implement it.

[totalgee]

The recent change improves the program change timing, it does now appear ~1 ms before the noteOn, but the noteOff thing you changed isn't working properly (it seems like maybe it's backwards from what it should be?). If I give it the same duration as the time interval, the noteOffs actually arrive at the same time or slightly after the noteOn (so each note is completely choked immediately after starting).

For this example:

set tempo 120
new midi default time(1/4)  note(0 2) gain(1) length(1/4) name(mid)

I get this:

[totalgee]

It would be possible to implement this yes, but it does indeed go against how everything else works. Do you think some systems might have a hard time if they keep receiving the program change message even tho it doesn't change? I'm not sure if most MIDI devices already filter for this or if they try to change anyways even tho the number is the same. If that's the case then this could be a clear reason to implement it.

I do have some VST synths (e.g. AAS Player) which seem to require "non-trivial time" to switch programs, something on the order of 50ms (just estimating... it may depend on the patch it's switching to?). And (for that synth at least) when you send the same (current) program number again, it unfortunately does seem to stop the currently playing note and cause an audible hiccup. But, as I say, this is probably different for each type of VST or physical MIDI instrument. Probably switching programs during a performance (like for every phrase or note) would not be a normal usage, for this reason. It happens that the M8 I'm using does support/require it and can perform the switch almost "instantly", but I don't think this is true in general.

It would be nice if there was a way (for those other types of synths that can't switch program quickly, and cause notes to be cut off when switching) to switch programs "occasionally", like a one-time program change message you could run, or before the start of a longer phrase. But probably not worth wasting too much time trying to solve this for my own weird use cases. ;-)

I tried doing something like the following (with two different "players" on the same MIDI channel, one only doing occasional program changes), but it seems it's also sending a note message along with the program change, but I couldn't really find something that made it switch "cleanly", maybe because it's also sending a note:

new midi default time(4) program([5 6]) name(midiProgChange)
new midi default time(1/4) wait(30) note([0 2 4 6] 2) length(1/4) name(midiNotes)

Note that at least the duration issue is not a problem in this example, because I'm not sending the same note repeatedly (on midiNotes), I'm changing it each time (0, 2, 4, 6). Also note that if I don't put the wait(30) the glitch is more audible when switching programs.

[tmhglnd]

And (for that synth at least) when you send the same (current) program number again, it unfortunately does seem to stop the currently playing note and cause an audible hiccup

I've implemented that the program change is only send when the number is different from the previous one. It will always be send once upon evaluating the code tho!

The recent change improves the program change timing, it does now appear ~1 ms before the noteOn, but the noteOff thing you changed isn't working properly (it seems like maybe it's backwards from what it should be?). If I give it the same duration as the time interval, the noteOffs actually arrive at the same time or slightly after the noteOn (so each note is completely choked immediately after starting).

I think the 1ms reduction on the duration might not be enough then. I've measured the timing and in most cases when I use new midi default time(1/4) length(1/4) the time between note-on note-off is about 499ms. But sometimes for me also it does arrive a bit too late cutting of the next note. I've subtracted 5ms now from the length, this gives a bit more room and seems to work well for me.

I tried doing something like the following (with two different "players" on the same MIDI channel, one only doing occasional program changes), but it seems it's also sending a note message along with the program change, but I couldn't really find something that made it switch "cleanly", maybe because it's also sending a note:

Yeah the midi instruments always send a note, similar to that the synth and sample always play a sound. When I designed this i didn't really consider the midi output to be used for other things then sending midi notes haha. Maybe I should consider this to be possible tho if it's really in the way of doing other things with midi.

[totalgee]

This sounds good, I try it out with your 5 ms reduction, I suspect that should work well, except maybe in cases where I want to do portamento and not send a note off until the next one has started, ha ha. (But in that case I can just set the length longer still) As we've seen, there will probably always be some weird cases where the timing gets messed, though, no matter what you do.

When I designed this i didn't really consider the midi output to be used for other things then sending midi notes haha. Maybe I should consider this to be possible tho if it's really in the way of doing other things with midi.

Regarding that, "MIDI output for things other than notes", besides program change, I can imagine wanting to send one or several CCs on a "track" with different timing than the notes, more like an LFO that's just running at its own rate, parallel to the note events.

[totalgee]

I tested the latest change. The program change (not sending when the value hasn't changed) seems to work well, thanks!

Regarding the 5ms shorter note durations, I don't see it being very reliable, I still get glitches. At 120 bpm and 1/4 notes (500ms spacing), sometimes the time between noteOn and corresponding noteOff is, in one example (in seconds, ranging from 0.485 to 0.504 when theoretically it should be 0.495 with your change):

0.494, 0.495, 0.494, 0.500, 0.497, 0.504, 0.485, 0.493, 0.494

I'm not sure if this is due to imprecision in the browser, or maybe even in the Windows loopMIDI virtual MIDI adapter. In fact I think the timing here seems much worse than before (not to do with note duration). It now seems very variable, if I play something like 1/16 notes it's quite unpredictable now over MIDI. (Update: I think it's maybe due to the loopMIDI thing...if I play to a "real" synth like the M8 it is much more stable, although I still get notes that overlap and sometimes cut off the next one).

[tmhglnd]

Regarding that, "MIDI output for things other than notes", besides program change, I can imagine wanting to send one or several CCs on a "track" with different timing than the notes, more like an LFO that's just running at its own rate, parallel to the note events.

Makes sense, I'll give it some more thought on how to make this (also don't want previous code to break of course)

I'm not sure if this is due to imprecision in the browser, or maybe even in the Windows loopMIDI virtual MIDI adapter. In fact I think the timing here seems much worse than before (not to do with note duration). It now seems very variable, if I play something like 1/16 notes it's quite unpredictable now over MIDI.

I don't know, I think i'll have to take a deeper look at how the playing of notes are scheduled in WebMIDI. I'm currently not doing anything complicated, originally the note-off was scheduled at the time exactly the duration of the note in milliseconds (so same as the time()) and I just subtracted 5, haha. Maybe I should manually schedule the note-off messages in some other way.