MIDI overhaul - interaction with the rest of Giada

Here I wanted to discuss "The Model", how the overhauled MIDI engine should interact with Giada. This needs to be established before we can proceed with GUI and polishing functionality.

First things first, the main change in new MIDI system is a multi-port capability. This means that each binding of a MIDI message with some action in Giada must include port information as well. Bindings are also foreseen to be more flexible in the future, allowing for multi-button combos or better handling of knobs, but for now, MidiBinding class represents a simple Note/CC filter, including a specified message sender. Methods are provided to return either integer or a boolean, and the latter should work well with all controllers I've seen so far. This class is already json-serializable, and this means it can be directly stored and loaded in giada.conf and projects.

giada.conf should hold all the "global" settings, which means all MIDI-related settings that are set in config window. This also includes global MIDI bindings, like metronome, play, volume etc. From these settings, midiDevice class will handle global bindings, midiPorts will fetch all selected ports and maintain connections with them, midiClock will do whatever is set concerning the sync. I think that all these modules will grab the settings they need, during init/reinit.

According to my idea, channels parse "musical" MIDI messages (such as notes, CCs, pitch bend) on their own. However channels should not "own" the information about MIDI routing from or to them, which I believe is currently the case. Internally, all MIDI routing will be handled as entries in midiDispatcher tables. This completely breaks the current state of affairs, where the MIDI input and output are the properties of a channel - but isn't practical when multiple inputs, multiple outputs and channel-to-channel connections are in place. Again, we go full flexibility.

Similarly, MIDI messages that trigger actions on channels and plugins (tweak parameters, start/stop channels, mute, arm etc) will be handled by midiController, and all these bindings should be stored in project files, as a part of.. midiController model, not channel model, whether that makes sense. midiController should also be notified of all channel state changes to provide feedback to controllers.

midiController should apply actions to channels and VSTs pretty much like midiDispatcher did so far, using glue. While we're at it, #384 pops into my head - as a rule of thumb, probably all saveable channel parameters could be midi-mappable.

Now, I don't know much about model, and I'm not sure if I understand that correctly - I see it as a more advanced object templates, but frankly I don't know why aren't these just a bunch of classes. :) So, I hope for @gvnnz 's thoughts, how do we make all that happen.

Thanks!

I was meant to post it a few days ago, but a kernel update went very wrong. Then all sorts of things went wrong, but what I'm trying to say is that there is no need to worry about my recent activity. Just like the last time, I expect a boost in MIDI development once I get some time off, between Boxing Day and New Year's eve. :)

Anyway, my backup system proved really reliable and it saved my note I wrote last time.

I see your point when you talk about moving information from channels to external components. Again, the json-based model should help with that.

When it comes to de-serialize stuff, it all begins from a json file which is parsed by m::patch (if it's a project file) or m::conf (if it's a global configuration file). Either way, the task of those functions is to read the json files and de-serialize them into objects: conf and patch respectively. They also do the opposite, from objects to json data (serialization).

At this point it really is a matter of how the components you designed interact with other threads. So, before going into a detailed description of how to put conf and patch objects into the model I need to understand how your new MIDI components interact with each other and the rest of the system, thread-wise. For example, I assume MidiBinding will be called from the RtMidi thread: will it share data with other threads such as the audio thread or the main (GUI) thread? What about other components? What type of data do they share?

If we are lucky, and I think we are if I understand your proposal correctly, all new MIDI components can become part of the model as simple objects and exchange data through a lock-free queue (we have one here).

Hi, hello and happy holidays! Finally got some spare time to tinker with Giada again and make a leap in this effort.

Honestly I never considered Midi Overhaul elements to be threaded, but in reality I think they ought to be treated as such. I'll try to explain how do I see it in this regard.

Thread 1: Message incoming from RtMidi

I assume that RtMidi already has a thread on its own that calls midiPorts callback function. Right now it returns to RtMidi after the message has been fully processed, which probably isn't the most fortunate solution. I'd draw a line at the point when Midi Message enters Dispatcher - it should have its own queue and its own thread, so the senders can not bother about the message after it has been sent.

Thread 2: Midi Dispatcher

This thread should "wake up" when a Midi Dispatcher queue is not empty. A hint would be appreciated how to handle it in a better way than polling. Registering and unregistering receivers in midiDispatcher could also be queued and processed by midiDispatcher on its own for the sake of integrity of its internal tables. Fortunately enough, neither message forwarding, nor reg/unreg operations do not return any important information, so can be safely queued. Note to myself: reg/unreg queues should have higher priority over the message queues.

Thread 3: MIDI / Sample channels

This obviously is an existing thread that should be modified a bit. AFAIK it has a MidiEvent input queue, that should be changed to MidiMsg queue. I expect this thread to parse the MidiMsg on its own and act accordingly. Remember that according to my plan, MidiMsgs passed directly to channels are not meant to control them, merely feed "musical" messages, like notes, CCs etc. Channels may send MidiMsg objects as well, that would be queued in midiDispatcher.

Thread 4: Midi Controller

Since I plan to make it a "brain" of otherwise dumb controllers, it definitely should have a thread of its own. This implies an input MidiMsg queue, an ability to output MidiMsgs to Dispatcher, but also be aware of the current playhead position (possibly a function called on each half-beat?), be able to manipulate Midi/Sample Channel properties and states somehow...

Thread 5: Midi Clock

Midi Clock module is not even prepared yet, and I'm not fully convinced whether it should have its own thread, or be a collection of methods gluing audio engine clock with Dispatcher. Its task would be to correct playhead position and tempo according to incoming clock messages, and send MIDI clock messages as the playhead moves. I don't expect anything more going on in there, however since I'm so good at making up threads, it could be a separate one as well, just idle for most of the time.

Things that do not need to be separate threads:

Midi Learner

This one is being called by a GUI at some point, and sets up an exclusive rule in MidiDispatcher to catch a MidiMsg incoming from any external port. Once this is done, it reverts a Dispatcher rule and calls a callback previously set up by a GUI. This is a solution based on old midiDispatcher learning method, and although seems overcomplicated, actually lets GUI cancel the learning process if requested. So, to sum up, I think MidiLearner is simple enough to be operated by GUI and Dispatcher threads directly.

MidiDevice

As a global bindings handler, it does nothing more than parsing a MidiMsg and calling an appropriate audio engine function - it probably is just too simple to be worth a hassle of transforming it into an independent entity with a separate thread. However, this module has been separated just in case someone decides it's actually a great idea to transform MidiDevice into a full-blown API operated by MIDI messages. Even in that case, I'm not fully convinced it must be a separate thread.

MidiPorts

This module merely provides bidirectional interfaces to all open ports, except of opening and closing ports, which happens rarely. I think that midiDispatcher can wait for the outgoing message to be enqueued in RtMidi, and conversely, RtMidi can wait a while for the message to be placed in MidiDispatcher's queue. There is no gain in providing an extra buffer at this stage.

So it seems that almost every module I came up with should be a separate thread. I'm not entirely sure if we'd benefit from making an exceptions to those few that do not need to be separate threads - perhaps for the sake of simplicity, all of these should be unified and converted to work as such?

Anyway, this made me think there's much to be done in terms of threading. I'll try investigating how exactly models work in Giada, how should I implement these queues etc.

Thanks for your patience and terribly sorry about the delay. On the bright side, my cat has survived a serious surgery and is doing much better so I can divide my attention to other things as well.

So concerning this queue implementation. I assume it's FIFO, which is what I need, but push and pop are names most often used with FILO, like stacks. Just wanted to make sure it's FIFO :)

Another thing is that most of the queues I need, such as Dispatcher input, Channel input etc, will be multiple producer, single consumer queues. Could it be easily adjusted for this case, or does it call for a completely different implementation?

Hey @tomek-szczesny , welcome back - and welcome back myself too :) It's good to hear from you (and your cat) and see you are making progress on this. I will be pretty busy with other Giada-related stuff in the next few weeks, but I can provide feedback of course.

WRT multithreading:

So it seems that almost every module I came up with should be a separate thread.

I'd prefer to avoid having one thread per module, if possible. RtMidi already runs its own, so it would be great if you could reuse it. If it makes sense of course. Multithreading is hard and adds a lot of complexity when it comes to synchronization and such.

So concerning this queue implementation.

I always forget the FIFO/FILO difference - that's a lock-free data structure where you push things back and pop them from the top. It's also single producer single consumer, so can't be used by multiple writers. I would keep it as is and add a new one that supports what you need.

Hi, and I hope you're doing great as well!

So if you suggest to keep the threads at minimum, I think we could go down to just a few:

RtMidi (Existing)
Sample / MIDI channels (existing)
midiDispatcher
midiController

midiClock can be reduced to a glue between Dispatcher and Audio Engine clock.

If it comes to synchronization, all communication between aforementioned threads is almost entirely done by dropping a MidiMsg object into a queue of some other object - so I guess a multi-producer FIFO queue would be enough to handle it all.

midiController would also need a queue for event reports from the channels - I figured it's much more efficient to gather that data rather than poll for it. Anyway the same multi-producer queue template would apply.

What I do not know is how to spawn these extra two threads, and how to include all this in the model.

push things back and pop them from the top.

That's definitely FIFO. First [that comes] In [is the] First [that pops] Out. :) I'll just make a new queue template to support multi-producer scheme. Right now I can't really tell what makes it single-producer, but I guess I'll find out.

monocasual / giada