search

free-audio / clap

Audio Plugin API
https://cleveraudio.org/
MIT License
1.71k stars 98 forks source link

midi controls extension #352

Open Bremmers opened 9 months ago

Bremmers commented 9 months ago

midi-info extension

abique commented 9 months ago

Hi, I'm not familiar with the MIDI info, is it part of MIDI 2.0? Given that we have raw MIDI 2.0 already do we need this? Isn't this already part of the MIDI 2.0 protocol?

Bremmers commented 9 months ago

"MIDI info" is just the name of the extension. A host needs this info from the plugin if if wants to support parts of Property Exchange which is part of MIDI-CI (CI = Capability Inquiry). MIDI-CI is part of MIDI 2.0, but it can also be implemented by MIDI 1.0 devices. It's built on sysex messages.

One of the Property Exchange resources is 'ChCtrlList', which is a JSON array describing all MIDI controls the receiver responds to. Here's a super-short one, as an example:

[ { "title": "Volume", "ctrlType": "cc", "ctrlIndex": [7], }, { "title": "Modulation Wheel", "ctrlType": "cc", "ctrlIndex": [1], }, ]

A sender (keyboard) receives this info from a receiver (DAW, synth etc.), so it can configure itself to work with the receiver. That's the thing MIDI-CI brings to the table: the sender knows things about the receiver, which is something new for both MIDI and CLAP.

abique commented 9 months ago

So the benefit of using this extension versus sysex is that you can query it on the main thread?

baconpaul commented 9 months ago

Here's the thing I don't quite understand @Bremmers (and it really is ignorance not challenge)

  1. The MIDI 2.0 CI spec is huge - I just went and looked - and this extension has a very small set of API points. Things like hand shakes, authority levels, etc... seem very mechanical, but things like Protocol exchange seem on the same level as the sub-set of the API in this extension.
  2. As Alex said we have midi 2.0 messages; what is the difference between this and them? Can we just generate this as midi-2.0 messages on the audio thread? What is different about having it as an extension?
  3. At an absolute minimum, the extension is misnamed. "midi-info.h" is "information about midi" whereas this is definitely "implementation of the midi 2 property exchange" so at least we should rename it to midi2-property-exchange.h and associated. And perhaps have a midi2-protocol-exchange.h extension parallel (or perhaps a midi2-ci.h which contains both
  4. I still think we should segregate out the note expression support into a separate draft extension which is not buried in midi and vst wrapper copies. I don't have time to do that right now but can propose something if you want.
Bremmers commented 9 months ago

@baconpaul

  1. The hand shakes etc. are for communication over a wire, where the other end may fail to respond. These problem don't exists at the host/plugin interface. It takes like 10 different sysex messages going back and forth to retrieve the info this extension is about BTW, so a higher level API makes sense I think.

The other 'big thing' besides Property Exchange is Profiles. Protocol negotiation was moved from MIDI-CI to the UMP protocol, I suggest you just ignore that bit.

  1. First, this is about meta information about MIDI messages (names etc.). Second, this information goes in opposite direction: from the MIDI receiver (the plugin) to the sender (the DAW, and with that the keyboard).

MIDI-CI is bidirectional. A MIDI-CI enabled piano keyboard which sticks to the old MIDI 1.0 standard would have two DIN cables, the second one is for received data from the DAW or synth it's connected to. But it's still just a piano keyboard that doesn't include sounds. In fact, such a device seems to be coming soon: https://www.gearnews.com/leak-korg-keystage-midi-2-0-controller-with-polyat/

  1. While I don't care about the name I think "information about midi" is a good way to think about this. It's not necessarily limited to property exchange. MultitrackStudio, for example, presents the user nice lists of MIDI controllers that actually work with the current instrument, including correct names. This obviously works with the built-in instruments, and now even with MIDI-CI enabled hardware instrument (https://www.kvraudio.com/news/bremmers-audio-design-adds-midi-ci-support-to-multitrackstudio-58816). It doesn't work for plugins because, well, a host knows nothing about a plugin's MIDI implementation. This extension fills that gap. Having to read the manual to see if a synth responds to ccWhatever is so 1983 ;-)

There was a PR for a profiles extension which I accidentally deleted two days ago . It's still here: https://github.com/Bremmers/clap/blob/Bremmers-midici-profiles/include/clap/ext/draft/midici-profiles.h It's much more complicated than it seems though, so I'm not in a hurry with that one.

  1. I agree with that. I couldn't find that in the wrapper repo. Can you point me to it?

IIRC we explicitly didn't do things like this because it can lead to problems if a plugin supports a certain feature but fails to advertise it correctly, or vice versa. But hosts need to know...

Bremmers commented 9 months ago

So the benefit of using this extension versus sysex is that you can query it on the main thread?

It also defines a way to get information out of a note input port. The information is going in opposite direction.

baconpaul commented 9 months ago

https://github.com/free-audio/clap-wrapper/blob/7cc84d49aebc1d3740f2ab9aa92e01b7555ec712/include/clapwrapper/vst3.h#L106

there's the wrapper code.

your comment #3 - "having to read the manual to decide mid cc whatever is whatever" - makes me think this really is tightly coupled with the midi param mapping somehow.

Bremmers commented 9 months ago

there's the wrapper code.

Hmm, I actually did find that file by myself, but stopped reading two lines before the good bit started. Pfff...

Looks good to me. If it ends up being a standalone extension it would need a host interface with a 'changed' method I think.

your comment #3 - "having to read the manual to decide mid cc whatever is whatever" - makes me think this really is tightly coupled with the midi param mapping somehow.

I see your point. I wonder why exactly we do have https://github.com/free-audio/clap/blob/main/include/clap/ext/draft/midi-mappings.h in the first place. @abique ?

baconpaul commented 9 months ago

Yeah my view, and again remember it is a woefully uninformed one so I'm being careful to admit that, is that we probably want extensions which

  1. Allow you to advertise note expression support (with the implicit assumption being that without it you take any)
  2. Provide midi property exchanges completely, with a name like midi2-property-exchange. This is a good start for that
  3. Provide a way for those properties to also associate with params for daws. midi1-cc-param-mapping is probably the right name for that.
  4. It might be that 2 subsumes 3; or 2 conflicts with 3; or 2 and 3 can and should safely co-exist but we should figure that out and document it if we have both; and figure it out and document why we dropped one if not

again: woefully under informed. I have never touched a midi 2 device.

abique commented 8 months ago

I see your point. I wonder why exactly we do have https://github.com/free-audio/clap/blob/main/include/clap/ext/draft/midi-mappings.h in the first place. @abique ?

It is an old thing. I don't see it being useful because it doesn't allow sophisticated mappings. We can probably throw it away.