MIDIDevice send_now() needed

[PaulStoffregen]

Latency sensitive messages like sendRealTime(0xF8) need the send_now() function to avoid waiting 200us at high speed or 1.5ms at full speed.

[doctea]

Thanks for looking into this! I'm sure there's probably something heinously wrong with what I've attempted here, but I seem to have worked around this in my fork here -- have only tested it with a couple of devices and only for 15 minutes or so, but its a definite improvement https://github.com/doctea/USBHost_t36

[doctea]

Don't know if anything has been done to the latest versions that may solve this...?

However I found that I'd accidentally merged the recent changes in to my fork, and found that meant that it wasn't compiling anymore for me. So I've recreated the repos and forked from the point where it works into https://github.com/doctea/USBHost_t36/tree/fixing-midi-clock in case this helps anyone else in the meantime.

[doctea]

After playing around some more with my hack that gets MIDI clock working, I've found that it interferes with other MIDI functionality.

If I'm playing MIDI notes and sending clock at the same time, the MIDI note on/offs more often than not get lost completely, leading to nothing playing, or stuck notes. If I disable my hack that gets the MIDI clock working, then notes work fine. Kinda frustrating!

What's even weirder is that I'm finding that if I connect to the Teensy from the host PC via serial monitor, where I'm echoing a log of all the notes being sent, then it improves the sending of the MIDI note on/offs immensely. Why would that be, that connecting to the Teensy over serial would be improving the reliability of the USB host MIDI data?!

I'd really appreciate some pointers on how to get this fixed. I bought a Teensy specifically to use MIDI USB in this way and this strange behaviour is really holding back my project now. Although I've learned a lot to get this far, I don't know enough to know whether I'm encountering new bugs in the USBHost library, or just implementing my fix poorly!

Any info, tips or help would be very much appreciated! :) Maybe there is even some kind of bounty system where I could donate to get this problem solved?

[PaulStoffregen]

I hope you can understand nearly all available time is currently going into chip shortage issues.

If I manage to look at this anytime soon (a pretty big "if"), time to play with the code and do testing will be very limited. And as you're experiencing, hacks to USB host require significant testing.

Is any sort of test case code available, plus a specific MIDI device or setup (or another Teensy in MIDI device mode running specific code), maybe on a forum thread? One or more good test cases would really help maximize the amount of work that could be done on this within the very limited time available.

[doctea]

Thanks @PaulStoffregen , understood. I'll try and put together a simpler test case somehow if that will help. The issue seems pretty obvious as soon as you connect more than one USB MIDI device and try and send them clock and note on/offs at the same time, though. I see how complicated it can be and hats off to you for everything so far :)

[doctea]

Well, after some more experimentation I seem to have worked around the problem I was having, although I'm still not sure exactly what was going wrong. Previously I was looping over each MIDI device, and sending clock and note on/offs where necessary within the same loop. If I restructure things so that I loop over each MIDI device to send note the on/offs, and then loop again to send all the clocks, then this seems to work. Maybe this will help someone else figure out what the underlying problem is or to solve the problem in their own code!

[insolace]

Could you provide full example code for both methods, and describe what to look for in the incorrect midi output?

On Jul 23, 2022, at 9:04 AM, Tristan Rowley @.***> wrote:

 Well, after some more experimentation I seem to have worked around the problem I was having, although I'm still not sure exactly what was going wrong. Previously I was looping over each MIDI device, and sending clock and note on/offs where necessary within the same loop. If I restructure things so that I loop over each MIDI device to send note the on/offs, and then loop again to send all the clocks, then this seems to work. Maybe this will help someone else figure out what the underlying problem is or to solve the problem in their own code!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[doctea]

@insolace Thanks for taking an interest. The project that I'm seeing the problem with is pretty huge now, but I'll try and find some time over the next month to distil it to the bare necessary code to replicate this problem.

In the meantime though, in case you wanted to get stuck in, the project is at https://github.com/doctea/usb_midi_clocker/ and this commit on this branch shows the difference between it working as expected, or getting very sporadic MIDI note throughput/stuck notes etc. The 'only' difference there is sending the recorded notes before or after sending the clock ticks. Seems to work flawlessly if sent before, barely works at all if sent after (or at the same time as the clocks in the same loop).

This is the fork/branch of the library that solves the original problem of the MIDI clock over USB to multiple devices being extremely stuttery/possibly garbled; I wouldn't be at all surprised if the problem I'm seeing with MIDI notes not working is because of a problem with that workaround that would otherwise be solved if /send_now()/ were implemented in a more proper way. So perhaps that is the better place to start.

What is really odd, and what makes me think that this may be a problem in the library rather than what I'm doing in my code per se, is that there is a stark difference when the IDE is connected to the Teensy serial monitor. If its not connected, then hardly any MIDI notes get through. But if it is connected to the serial monitor and debug information is being sent (flushed?) regularly then things seem to almost work 'properly', as if the USB serial connection is influencing the USB MIDI streams too.

I've also just checked out the MIDI-Router project on your github -- very cool -- not a million miles off what I'm trying to do either (although my project is suffering from rather a lot of feature creep!). Am wondering why you didn't run into the same problems I have; the problem with USB clock being very unusable when sending to multiple USB devices bit me as soon as I connected multiple USB MIDI devices and tried to sync them with clock in the most naive of implementations --- although it seems like this was perhaps a problem introduced into the USBHost_t36 library more recently than the MIDI-Router project? And the problem with MIDI notes being unreliable only seems to present itself when I'm playing pre-recorded notes as well as sending clock, but seems fine when simply passing notes through from one device to another, so maybe you aren't hitting the same conditions. Or does your project not deal with MIDI clock at all? (I've looked through a fair bit of the code -- nicely structured and written btw -- but didn't spot any MIDI clock stuff, but maybe I missed it?)

Sorry about all the text - hope something in the piques your interest anyway!

[insolace]

Hi Tristan,

Thanks for the thoughtful reply.

MIDI router has been untouched since I started working at Keith McMillen Instruments, but I’ve wanted to revisit it and more specifically the Teensy MIDI library for a while. Back in 2020, MIDI router was working fine with clock being sent to multiple devices, so I will need to dust it off and do some further benchmarking to see if I can observe what you are running into and reproduce the issue.

That said, your description of the serial monitor fixing things makes me suspect that the problem is with stuffing the USB data packet and sending it regularly. If serial data is triggering a send then the library probably needs to regularly check if there is MIDI to be sent. At KMI we check the buffer every millisecond and send if there’s any data, regardless of how full it is.

On Jul 26, 2022, at 4:38 PM, Tristan Rowley @.***> wrote:

 @insolace Thanks for taking an interest. The project that I'm seeing the problem with is pretty huge now, but I'll try and find some time over the next month to distil it to the bare necessary code to replicate this problem.

In the meantime though, in case you wanted to get stuck in, the project is at https://github.com/doctea/usb_midi_clocker/ and this commit is the difference between it working as expected and getting very sporadic MIDI note throughput/stuck notes etc. The 'only' difference there is sending the recorded notes before or after sending the clock ticks. Seems to work flawlessly if sent before, barely works at all if sent after (or at the same time as the clocks in the same loop).

This is the fork/branch of the library that solves the original problem of the MIDI clock over USB to multiple devices being extremely stuttery/possibly garbled; I wouldn't be at all surprised if the problem I'm seeing with MIDI notes not working is because of a problem with that workaround that would otherwise be solved if /send_now()/ were implemented in a more proper way. So perhaps that is the better place to start.

What is really odd, and what makes me think that this may be a problem in the library rather than what I'm doing in my code per se, is that there is a stark difference when the IDE is connected to the Teensy serial monitor. If its not connected, then hardly any MIDI notes get through. But if it is connected to the serial monitor and debug information is being sent (flushed?) regularly then things seem to almost work 'properly', as if the USB serial connection is influencing the USB MIDI streams too.

I've also just checked out the MIDI-Router project on your github -- very cool -- not a million miles off what I'm trying to do either (although my project is suffering from rather a lot of feature creep!). Am wondering why you didn't run into the same problems I have; the problem with USB clock being very unusable when sending to multiple USB devices bit me as soon as I connected multiple USB MIDI devices and tried to sync them with clock in the most naive of implementations --- although it seems like this was perhaps a problem introduced into the USBHost_t36 library more recently than the MIDI-Router project? And the problem with MIDI notes being unreliable only seems to present itself when I'm playing pre-recorded notes as well as sending clock, but seems fine when simply passing notes through from one device to another, so maybe you aren't hitting the same conditions. Or does your project not deal with MIDI clock at all? (I've looked through a fair bit of the code -- nicely structured and written btw -- but didn't spot any MIDI clock stuff, but maybe I missed it?)

Sorry about all the text - hope something in the piques your interest anyway!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[doctea]

Back in 2020, MIDI router was working fine with clock being sent to multiple devices, so I will need to dust it off and do some further benchmarking to see if I can observe what you are running into and reproduce the issue.

Would be very interested whether you run into similar problems, and whether it was a new bug introduced to the USBHost_t36 library when the bandwidth optimisation was added, which is my hunch at the moment. Thanks again for the interest and discussion :)

That said, your description of the serial monitor fixing things makes me suspect that the problem is with stuffing the USB data packet and sending it regularly. If serial data is triggering a send then the library probably needs to regularly check if there is MIDI to be sent. At KMI we check the buffer every millisecond and send if there’s any data, regardless of how full it is.

I /think/ that's what the existing USBHost_t36 code tries to do, if I'm understanding it right -- by setting a 200us or 1.5ms send timer set when any data is added to the buffer. But for some reason, that doesn't seem to work as-is for clock (the result is that the Beatstep, for instance, seems to randomly jump around randomly! - as if there are pauses where nothing is sent, followed by hundreds of clocks being sent instantly) - my hack forces a send immediately if a clock message is added to the buffer.

I'm not familiar enough with either USB or the programming pattern used by USBHost_t36 to understand out what the implications of that strategy might be, or whether there are any bugs in the current implementation of it, though.

I also wondered if perhaps it was something to do with MIDI Running Status, as from what I can tell from the USBHost_t36 code, 4 bytes are added to the buffer for every 1 byte of clock message. I guess this is because USB requires packet sizes that are multiples of 32bits, or something like that. I thought those extra bytes might be getting misinterpreted, either because of Running Status, or just because there are unexpected 0x00 bytes in the MIDI stream interrupting the note on/off messages. But I didn't get any positive results when I tried to adjust this (by changing which byte was used for the clock message, or by filling the extra byte with a clock).

With my modification of the library and being careful about when I send clock and note on/offs, its working well enough for me now that I can continue with the project, but all this behaviour does seem a bit strange! :)

[insolace]

Still haven’t had tile to dig into this.

Per the USB MIDI spec, all USB midi events are 4 bytes regardless if they’re clock or 2 or 3 byte channel events.

On Jul 30, 2022, at 11:05 AM, Tristan Rowley @.***> wrote:

 Back in 2020, MIDI router was working fine with clock being sent to multiple devices, so I will need to dust it off and do some further benchmarking to see if I can observe what you are running into and reproduce the issue.

Would be very interested whether you run into similar problems, and whether it was a new bug introduced to the USBHost_t36 library when the bandwidth optimisation was added, which is my hunch at the moment. Thanks again for the interest and discussion :)

That said, your description of the serial monitor fixing things makes me suspect that the problem is with stuffing the USB data packet and sending it regularly. If serial data is triggering a send then the library probably needs to regularly check if there is MIDI to be sent. At KMI we check the buffer every millisecond and send if there’s any data, regardless of how full it is.

I /think/ that's what the existing USBHost_t36 code tries to do, if I'm understanding it right -- by setting a 200us or 1.5ms send timer set when any data is added to the buffer. But for some reason, that doesn't seem to work as-is for clock (the result is that the Beatstep, for instance, seems to randomly jump around randomly! - as if there are pauses where nothing is sent, followed by hundreds of clocks being sent instantly) - my hack forces a send immediately if a clock message is added to the buffer.

I'm not familiar enough with either USB or the programming pattern used by USBHost_t36 to understand out what the implications of that strategy might be, or whether there are any bugs in the current implementation of it, though.

I also wondered if perhaps it was something to do with MIDI Running Status, as from what I can tell from the USBHost_t36 code, 4 bytes are added to the buffer for every 1 byte of clock message. I guess this is because USB requires packet sizes that are multiples of 32bits, or something like that. I thought those extra bytes might be getting misinterpreted, either because of Running Status, or just because there are unexpected 0x00 bytes in the MIDI stream interrupting the note on/off messages. But I didn't get any positive results when I tried to adjust this (by changing which byte was used for the clock message, or by filling the extra byte with a clock).

With my modification of the library and being careful about when I send clock and note on/offs, its working well enough for me now that I can continue with the project, but all this behaviour does seem a bit strange! :)

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[doctea]

Still haven’t had tile to dig into this. Per the USB MIDI spec, all USB midi events are 4 bytes regardless if they’re clock or 2 or 3 byte channel events.

Thanks for clarifying :)

[doctea]

To follow up on this -- for whatever reason, updating to the latest version of the library (as opposed to a fork made at the time this issue was created) seems to have resolved the issue I was seeing with USB MIDI clock not working correctly when multiple devices were being used, which was what I originally needed a working _sendnow for.