search

christofmuc / KnobKraft-orm

The KnobKraft Orm - The free modern cross-platform MIDI Sysex Librarian
GNU Affero General Public License v3.0
208 stars 26 forks source link

Make a VST for KnobKraft #46

Open christofmuc opened 3 years ago

christofmuc commented 3 years ago

This came up multiple times, now with JUCE 6 it should be easier as it was with the old CMake system.

Question would be what features are possible in a DAW, and what makes sense.

Would you create one KnobKraft per project, or one per Track? Would the track be then just the patch selection? How to detect the synth?

I think I don't know enough about DAWs to understand the use case yet.

christofmuc commented 3 years ago

It seems this is a hot topic for the past... 15 years?

Then there is this comment: " i believe you can add VST librarian in to Reason VST rack (such as soundquest midiquest) and it will send sysex as long as you are using midi port which is not occupied in Reason it self. It should work and of course depending on what you want to do. Sending sysex directly from Reason or recording sysex directly in to reason sequencer as in other DAW apps does not work ATM."

What good would that be for?

I found a comment here in that CTRLR is doing it: https://www.kvraudio.com/forum/viewtopic.php?t=409829&start=15

markusschloesser commented 3 years ago

the work around of using a separate midi port for direct sysex communication with a synth is used quite a lot, e.g. DEXED does this when you wanna sync it to the dx7. (https://github.com/asb2m10/dexed/wiki/Dexed-as-a-DX7-editor) However at least on Windows, not even a multi-client capable midi interface driver will allow you to select the same part twice, when it's coming from within the same program (see: ) One can use work arounds for that as well like loopbe or iConnectivity inbuilt, but it's complicated. (https://www.gearslutz.com/board/showpost.php?p=14695996&postcount=98 ) Edit: seems sysex in vst is not possible at all? https://www.gearslutz.com/board/showpost.php?p=14694597&postcount=92 However Microsoft Pete says it should be possible??

christofmuc commented 3 years ago

The VST3 API at least suggests you should be able to send events of type sysex, but yes, the folklore seems to indicate that this is mostly dysfunctional.

I tried plugging the old CTRLR VST into my Ableton Live lite, and I could not get any communication with my Matrix 1000 to work, even though CTRLR indicated it had opened the ports alright (it does the side-interfacing thing). I am pretty surprised, but having never worked with a DAW properly my ideas about these might be naive.

You can run KnobKraft alongside the DAW as a standalone program - wouldn't it suffice to have a VST plugin that can receive messages from the standalone software about which patches are selected, like some kind of interprocess communication? The point is to be able to store and recall the patches for a specific project, right. And if I can't inject sysex messages into a MIDI clip, the whole exercise seems to be a bit futile. The sysex tracks, however, seem to be very DAW specific and not part of the VST API.

I'm new to this DAW stuff, but I am not sure the effort is worth the possible result...

christofmuc commented 3 years ago

Ok, wild idea: Ableton supports program change, bank change, sub bank change commands in a MIDI clip. Now, those commands are mostly useless when you do no longer know which bank had been loaded into a specific synth, or have used the program slots in the meantime.

A KnobKraft VST could act as a MIDI filter for program change messages - it could just remove them so they are not sent to the device, but forwarded into KnobKraft. KnobKraft then has a mapping ("Program List") for that song/Ableton project that will trigger a sysex update of the attached song. KnobKraft then can return the program location of where the patch could be found in the device, and Ableton sends this.

Detailed design idea:

So, duties of the KnobKraft standalone:

  1. Have program lists (prerequisite to #9 as well)
  2. Have an API that can answer the following questions:
    • Get me a list of all program lists available
    • Get me the list and program numbers of the patches in a specific program list
    • Load a specific program into a device
    • Load the whole program list into the devices, storing them at certain program slots so it can be addressed with regular program change commands
    • Get me the real program change command for a patch from the program list
    • Get me a mapping table Program List -> Program -> real program change message that the VST can use to function when the standalone is not running.

Duties of the KnobKraft VST:

  1. Launch KnobKraft standalone if it is not running
  2. Display list of Program Lists available
  3. Display list of Programs in Program List
  4. Load a specific patch into a synth
  5. Load all patches from the Program List into the synths
  6. Receive program change messages, do a lookup via the API and create the correct program change message

  7. Store the name of the program list used in the VST state

    The split into the standalone and the VST allows for easier development, and also allows that once the synths are prepared for a project, to close the KnobKraft standalone should it fight for MIDI ports with the DAW. There are obviously workarounds like above when you setup virtual MIDI ports which merge, but this seems to be part of the overall problem.

markusschloesser commented 3 years ago

I already run kk as a separate program! :-) Mostly on the laptop, which has touch and is nearer to the synths. It is synced to the main pc. Which leads me to a couple of ideas:

  1. the Interprocess communication: great idea, don't know how sandboxed stuff is (especially on the OSX side). With regards to HOW, my favorite would probably networked or midi via rtpmidi (OSX and IOS inbuilt, for free on Windows, inbuilt in iConnectivity midi interfaces). With that it doesn't matter if the full kk instance is running on the same pc or another. And it's not your responsibility to implement / invent something new. But might make the whole syncing process harder. mmh 🤔 Will need to think about that. What kind of protocol did you have in mind?
  2. program change not necessary. A vst stores its state when saving the project/song, and recalls it when loading. With that you don't need to filter (messy) and can just restore the state
  3. per track vst / per synth vst: most of the time you have one (or multiple) tracks PER HW synth. Each needs to send its own (synth specific) stuff. So you have multiple KK VSTs talking to ONE main KK?
christofmuc commented 3 years ago

Not sure if I follow you - I was thinking probably simpler... in terms of technical feasibility.

My understanding is that yes, I can put the state (patch sysex) into the VST state, but how would it get to the device? we'd need KK standalone to do that, so the VST could just ask the standalone KK to send "this patch to that synth", and doesn't need to contain the information on its own.

The RTPMidi can already be used, it should be transparent to KnobKraft what kind of MIDI interface it is (also virtual and loopback MIDI interfaces should already work).

markusschloesser commented 3 years ago

Not sure if I follow you - I was thinking probably simpler... in terms of technical feasibility.

?

My understanding is that yes, I can put the state (patch sysex) into the VST state, but how would it get to the device? we'd need KK standalone to do that, so the VST could just ask the standalone KK to send "this patch to that synth", and doesn't need to contain the information on its own.

I select Patch 67/"awesome sound" (with ID 12234356) in the vst. the vst saves that info. When opening the project, the vst automatically recalls Patch 67/"awesome sound" (with ID 12234356) and sends that info to the fullblown kk, which then sends the sysex of that patch to the synth.

The RTPMidi can already be used, it should be transparent to KnobKraft what kind of MIDI interface it is (also virtual and loopback MIDI interfaces should already work).

I use already rtpmidi from the laptop to talk to the synths. But I meant for communication between the vst kk and the real kk

christofmuc commented 3 years ago

I select Patch 67/"awesome sound" (with ID 12234356) in the vst. the vst saves that info. When opening the project, the vst automatically recalls Patch 67/"awesome sound" (with ID 12234356) and sends that info to the fullblown kk, which then sends the sysex of that patch to the synth.

Yes, that'd be possible. And each instance of the VST would be responsible for one track with one instrument?

The RTPMidi can already be used, it should be transparent to KnobKraft what kind of MIDI interface it is (also virtual and loopback MIDI interfaces should already work).

I use already rtpmidi from the laptop to talk to the synths. But I meant for communication between the vst kk and the real kk

Ah, great! I would probably not even use MIDI between the VST kk and the real kk, no need. I can just do a proper interprocess call, JUCE as code for that. As both are running on the same machine, this could be simple.

markusschloesser commented 3 years ago
  1. it seems if vst, then it has to be vst3 because vst2 is not gpl compatible
  2. https://www.kvraudio.com/forum/viewtopic.php?t=538083 (mostly around problems with CC and vst3 but also other stuff like licensing and sysex)
markusschloesser commented 3 years ago

couple of links/info I stumbled upon while researching other (getting the fs1r working with the mackie c4 thru ableton):

  1. https://forums.steinberg.net/t/vst3-and-midi-cc-pitfall/201879/30
  2. https://forums.steinberg.net/t/how-to-save-parametres-correctly-vst3/202004/
  3. https://forums.steinberg.net/t/plugin-to-control-non-midi-devices/201935/
  4. https://forums.steinberg.net/t/generating-midi-output-repeating-a-loop/201900/
  5. https://forums.steinberg.net/t/midi-program-change/201860/
  6. https://forums.steinberg.net/t/clarification-of-parameter-handling-in-vst-3/201914
  7. https://jamba.dev (sounds complementary or even replacery to JUCE)
  8. https://forum.juce.com/t/vst3-midi-plugins-wont-load-in-ableton-live/36323/15
  9. https://forum.juce.com/t/add-support-for-sending-midi-ccs-out-of-vst3-plugins/35781/90

Things I learned:

You do need CC for KK, right? (i think I saw some CCs in the midi log)

All this imho clearly leads to doing as less as possible in the VST plugin and try to talk to the full KK in order to be able to do anything. That approach still needs a multi-client capable Midi-Interface (standard in OSX, for Win: RME, iConnectivity, probably more, didn't bother to find out)

christofmuc commented 3 years ago

The CC problems mentioned are more for interweaving notes and CC, that's not what we do. But yes, I think the lightweight VST with communication to KK is a faster and safer way, then you only need to solve potential MIDI interface conflicts (which is as far as I understand already the problem with all CTRLR panels, and other editors).

What would be a minimal viable UI for a VST?

mslinn commented 1 year ago

Pro Tools users would prefer an AAX version, instead of using AAX-to-VST bridge/wrapper to connect with a VST.

christofmuc commented 1 year ago

Good to know! JUCE as our C++ framework I think supports AAX.