Support for ON/OFF type CC midi events instead of value based midi events

[jesusyoshi54]

The mapping of MML to midi has general inconsistencies when it comes to certain CC events. These are the binary state single param events such as sustain, legato, or portamento. In SM64 and other MML, these events are handled by using two different cmds, an on and off, with no parameter to describe it. Currently to describe a CC event, the arguments of the MML equivalent can only inherit the value of the CC, but this is not valid for binary state ones, as they will have values of 127 or 0 usually, while in MML it needs to map to an on or off. There is also the issue of cases where MML cmds are comprised of multiple CC events. This is the case in portamento, where enabling, time and target note are all different events, while portamentos themselves are a single MML macro. This results in an impossible mapping.

[sauraen]

Thanks for the post!

MML portamento does not have a direct equivalent in standard MIDI files. The closest would be to use one CC to enable portamento, use another to set the time for the next sweep, play one note, and then play another while the first is still on. However, even in VSTs which have portamento, which is really only more full-featured synth plugins, this MIDI mapping isn't always supported. Portamento is also not used in vanilla music sequences in any MML game to my knowledge--the composers simply used pitch bends, as that was an easier way to create the same effect in their MIDI sequencer (usually Cubase).

Legato also is not used in vanilla music sequences, though I can see it being useful in some cases.

I'm not aware of "sustain" (as in MIDI sustain pedal, not the envelope sustain volume) being in any MML variant.

SEQ64 does support import and export of MML commands which correspond to multiple CCs, for example vibrato envelope commands in OoT. The CCs simply have to be on the same channel and at the exact same time, and they will be merged on import. I have not tried creating a mapping of CCs to the portamento commands; if you try this and run into problems, please let me know.

I should also mention that the portamento MML commands have an issue such that they actually aren't properly supported in SEQ64 (for anything, even MML binary <-> text), and ultimately can't be properly supported. One of their parameters is a var sometimes and a u8 sometimes, depending on the value of another parameter. Since var can change in length, parsing the command would have to change based on the contents of the command. This would be difficult to implement because SEQ64 handles MML syntax through the ABI definition--commands are not hardcoded. So I'd have to build a system where the user, through the ABI GUI, can set up any parameter's datatype to be dependent on the value of any other parameter. This can quickly turn into needing some sort of scripting support in the ABI definition (type = 'u8' if (param1 & 0x80) else 'var'), which is out of scope of the project. I got lucky that the vanilla portamento commands don't use all the supported configurations of the command.

So anyway, it would be possible for me to add functionality for the user to define a MML command to correspond to only a particular value of a particular CC. But since this would just be for legato, I'm not sure it's worth it. You could just have legatoon connected to CC 65 and legatooff connected to CC 66. Did that not work? You'll have to change their action to CC or CC Group, add a parameter with datatype constant and value 127 and 0 respectively, and change the parameter's meaning to CC and the appropriate CC number.

[jesusyoshi54]

I was unable to get legato to work. I consistently got this message from the console output: MIDI uses CC 68 (first use value 127), not mapped to any command in Audioseq. This is despite using the attached image mapping for legato

So far it appears that legato is the only one that has this issue, I did look into it more and it seems the other ones can be dealt with.

On the topic of sustain, that does exist in a conventional way, but I did find that it actually did map nicely unlike legato does because this one uses the same cmd with an arg much like MIDI does. I recorded an example of what sustain sounds like when used here: https://vocaroo.com/12W6AogyAc0q

Portamento is used not used in sequences, but it is used in sfx. There is no good way to preview it though, nor do the args map to any midi CC. I did try to create a mapping but because the args are above 127 it does not work.

[jesusyoshi54]

I believe this issue with legato, and also the one with portamento is a result of trying to map a CC or CC group onto a track. MIDI does not have track events like MML does, only channel events. When trying to add in a track event and exporting to MIDI I get this error: "CC or CC Group in somewhere other than channel header!"

This means it is impossible to store track events inside of the MIDI format. Ultimately this does not affect any sequences (that I know of), but it does affect SFX in SM64 at the least.

In terms of what this would make not possible for all MML events it would be: Legato Layer set inst layer portamento on layer portamento off layer pan

[sauraen]

Right, sorry, I forgot that those were track (layer) events rather than channel events. MIDI does not have a concept of layers (MML tracks) at all; the only note-specific commands are note on, note off, and polyphonic aftertouch. Unless we want some janky mapping of portamento commands to polyphonic aftertouch--and this will be a pain to edit in most DAWs--this is a limitation of MIDI, not of SEQ64.

SEQ64 is not intended to be able to convert arbitrary MML SFX into MIDI, simply because MML is a much more powerful format--these layer commands aren't the main problem, all of the programming stuff is. Of course, you can edit a specific jingle from the SFX sequence down into its own sequence, convert that to MIDI, edit it in your DAW, import it back as MML, and copy the channel / layer data back into the SFX sequence. But if it's just a few notes, it's probably easier just to write it by hand in MML assembly.

[sauraen]

There is one other option--have MIDI tracks map to MML layers/tracks. So for example, you'd have four MIDI tracks on channel 9, where each track corresponds to one MML layer, with one note at a time and support for the layer commands like legato and portamento. But, I don't think anyone is going to want to edit music in their DAW this way.

[sauraen]

Regarding sustain--do you mean the MML command called sustain? That is supposed to change the sustain volume of the envelope, not act as a MIDI sustain pedal. Those are somewhat similar but not the same thing. Are you sure that it is doing the latter (ignoring note offs and having the amplitude envelope continue as if you continued to hold down the note, and then when sustain is set to 0 triggering the note offs and the transition to the release cycle)? If so, this is a bug in both SM64 decomp and OoT decomp, as well as both Simon Lindholm's and my command info.

[jesusyoshi54]

Yeah it is labelled chan_setsustain in SM64 decomp, it does not affect the sustain volume of the envelope at all, but instead only acts after the note is released, and then sustains volume the same as a MIDI pedal would. In the example I posted, I am playing a scale of half notes, first without sustain and then with sustain. As you can hear, the note continues after when I let go of it and well into the second note. The exact behavior that I gather after reading the code for it is that the ASDR release state starts to decay, but if sustain is set, it is instead set to the chan sustain volume after releasing the note, and just holds it there indefinitely (or at least until the note pool clears it for some reason or other). It behaves the same as a midi sustain pedal would.

As for ASDR as it is understood in a conventional means of how the note volume behaves as you hold down the key, that is entirely decided by the envelope pair of s16s defined in the bank, or pointed to inside of the MML. There is no defined specific ASDR points at all, but it is just a set of volume keyframes to reach WHILE the note is held. After it is released, it goes to the volume set by the sustain cmd instantly, and then once sustain is gone, it will decay to zero over the time defined by "release_rate"

[sauraen]

Thanks for the info, I have updated the command name and description in the SEQ64 ABI files, and also forwarded this info to the work group on decomp Music Macro Language support so it can be corrected in both decomps.

then sustains volume the same as a MIDI pedal would

This is not how MIDI sustain pedal works. MIDI sustain pedal means it ignores note off until the pedal is released--it continues whatever envelope the note would have if you held it. Also, MIDI sustain pedal should transition to the release phase (from the current envelope volume) when the pedal is released if the note is not still being held--in this implementation, what happens if the sustain value is set to 0 while the note(s) are being held? Does it transition to release, or just set the notes' volume to 0, or ignore the sustain value change completely?

I know how envelopes work--based on the description for this command, I was assuming it would either change the current envelope point (and simply assume that was always the "sustain" phase), or actually move to the last point before the envelope command which tells it to hold the last level and modify that value. All of the command names/descriptions in Simon's docs and therefore in SEQ64 were decided upon before variable naming was done in the decomp, so all anyone would have seen was that this set some value in an unknown struct to do with envelopes.

[sauraen]

Based on feedback from Simon Lindholm and MNGoldenEagle, I've mostly reverted the changes above. The command is indeed an envelope-related command, and doesn't have anything to do with MIDI sustain pedal.

relsustain - Adds a sustain phase to the envelope release. At key off, volume
 falls at the release rate to the sustain value. Then it holds at the sustain value
 for 128 ticks, and then it falls at the release rate to 0. Changes to the sustain
 value while the note is in the sustain phase are ignored, so this command
 cannot be used like MIDI sustain pedal (releasing the pedal does not stop
 the note).