search

dwhinham / mt32-pi

🎹🎶 A baremetal kernel that turns your Raspberry Pi 3 or later into a Roland MT-32 emulator and SoundFont synthesizer based on Circle, Munt, and FluidSynth.
https://twitter.com/d0pefish
GNU General Public License v3.0
1.28k stars 79 forks source link

More compatibility issues with MIDI messages encoded with Running Status #26

Closed thorr2 closed 4 years ago

thorr2 commented 4 years ago

I have noticed that when songs start right away with notes (or if you skip around in a song) the sound is silent for a second or two so you miss the beginning of the music. This is using my PC or my Mister. It doesn't matter. Any idea why and can this be fixed? Thanks!

thorr2 commented 4 years ago

Just an update, I noticed when scrolling around a song on my PC the LCD screen said Midi Restart and the sound was silent during this time, but the notes on my PC kept showing as playing. I think the Midi Restart takes a few seconds to kick in and that might be causing the missing notes.

dwhinham commented 4 years ago

Can you give details of the software you're using to play/skip around and some example songs?

"MIDI restart" is just a warning and instantaneous, it just means that an incomplete message was received and it's reinterpreting the received data as a new message. This makes sense if you're skipping around because you're interrupting the MIDI stream.

thorr2 commented 4 years ago

Thanks. On my PC, I am using this MIDI player: http://falcosoft.hu/softwares.html#midiplayer You need to click on the bars in the upper left corner, then SysEx Options - Enable SysEx in files, and reset type - NoSysEx.

The problem seems to be affected by MIDI files that contain SysEx messages for the MT-32. In the MIDI player, they are detected and identified as "MT32" in the left side of the Midi Player's LCD in Windows. Here are some examples: http://www.midimusicadventures.com/queststudios/midi-soundtracks/complete-soundtracks/

For example, the King's Quest V soundtrack, it starts ok, but if you click and drag the progress bar to later in the song, it does the MIDI restart with no notes sounding while they are playing.

On my Mister, when I play the Colonel's Bequest (a Sierra DOS game), the beginning of the music gets cut off and shows MIDI restart on the LCD. This happens throughout the game at various places. I believe it is caused by MT-32 SysEx messages followed by immediately by MIDI notes. The MIDI files for this are at the above link, but they play fine because of the pause before the notes in the MIDI file. I didn't try it in DOSBox, but it may have the same behavior. There is a question at the beginning of the game to start playing it and the answer is usually the one in the upper left corner, otherwise you can google Colonel's Bequest Fingerprints to see an image to answer the question. It also happens on Monkey Island 2. Pressing Esc to skip to new music at different stages during the intro causes it to miss the first few notes.

Thanks!

dwhinham commented 4 years ago

Okay, there's a few things going on here!

This MIDI player is trying to be "clever" when you fast-forward. What it does is it scrubs through all of the MIDI events in the song, sending all of the parameter changes and SysEx it encounters - EXCEPT note events - until it gets to the point where you wanted to seek to. Then it resumes sending notes, and you begin to hear them again - this is presumably a feature that ensures all of the parameters are applied properly like panning and volume so that nothing gets missed.

The fact that the keyboard visualisation in the player is showing the notes being "played" right after you seek is probably causing some confusion here, because in reality, they aren't. I routed this player through a MIDI debugger so I could watch the MIDI traffic and the program is clearly stripping out Note On/Offs while it's seeking. The delay you're experiencing is this particular player program filtering through the hundreds of events and sending them as quickly as it can before getting to the point you seeked to, and not an mt32-pi bug.

As for MIDI restart - I'm not getting this at all while seeking around in Kings Quest V using this software, so I think we have a different unrelated issue here, possibly related to #25.

I've done a significant rewrite of the MIDI parser again, and there's a test build here: https://github.com/dwhinham/mt32-pi/actions/runs/252614148

Please try it out and let me know how it works for you!

thorr2 commented 4 years ago

Thanks. I tried the new test build and it is a significant step backwards. I am getting Unexpected MIDI status messages quite often and it is dropping lots of notes. Kings Quest V has problems from the beginning without moving the slider. The Mister has major issues as well. Long notes are short like staccato.

I assumed that skipping around in the MIDI file was doing what you described, but that was an easy way to show the behavior. I will test with my Sound Canvas on the Mister and see if it is also missing notes at the beginning of songs. It may be a Mister problem.

thorr2 commented 4 years ago

After testing with my Sound Canvas and Mister, I am pretty sure that it is a Mister problem. I am seeing some of the same behaviors. For now I guess you can close this issue. Thanks for your help! I would be happy to further test new versions, especially if you provide a way to update it via WiFi. ;-) Like I said, the latest test build is not good, so if you plan to use the new code let me know and I can try it. Thanks!

dwhinham commented 4 years ago

No worries! I still can't really afford a MiSTer setup right now so I guess we'll just have to hope the developers working on the MIDI stuff can help.

I'm a bit confused about where the MiSTer comes into play in this issue; are you running that MIDI player software on it? Or just the original DOS games?

Does the test build not work properly at all? Not even with the basic ScummVM on PC Monkey Island test, or playing your Pink Panther tune?

thorr2 commented 4 years ago

I am using the Mister with the ao486 core that hardware emulates a 486 DOS computer. I am only playing DOS games on it with a Roland UM One MK2 USB to MIDI adapter that plugs into the MT32-Pi's MIDI in port. For a basic Mister setup, all you need is the DE10 Nano that costs about $130 from Digikey. That is in the U.S. You can probably get it locally for around the same price.

I tried the test build with both Windows and the Midi player I linked above, and with the Mister using DOS games. In both cases, it had major problems. It might be related to the previous problem we discussed with the MIDI port not working vs. USB that was fixed once you implemented the full MIDI spec.

dwhinham commented 4 years ago

I broke down and purchased a UM-ONE Mk II and I can confirm sustained notes are being cut short in the test build. I'll be back with an update soon. 😃

thorr2 commented 4 years ago

Cool! Thanks!

On Thu, Sep 17, 2020, 7:41 AM Dale Whinham notifications@github.com wrote:

I broke down and purchased a UM-ONE Mk II and I can confirm sustained notes are being cut short. I'll be back with an update soon. 😃

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dwhinham/mt32-pi/issues/26#issuecomment-694282085, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVRPLQDUJ4XZWEU6N5JVVLSGINY3ANCNFSM4RKM26FQ .

dwhinham commented 4 years ago

After playing with the UM-ONE Mk II I discovered more flaws in the MIDI parser.

  1. UM-ONE sends Active Sensing packets even if the host PC does not, and I'd broken the logic that handles Active Sensing in the test build, so notes were being silenced - fixed.
  2. UM-ONE is a very clever interface, it seems to be able to not only compress MIDI data using Running Status, but may also be able to adaptively re-order messages for greater efficiency when the MIDI bandwidth is nearly saturated, allowing it to compress even more. It was sending Running Status packets for simple 2-byte messages and the parser was missing a check for this (the current release of mt32-pi only handles Running Status properly for 3-byte messages).

  3. Fast-forwarding in that MIDI Player app "too far" will completely flood the receiving device with messages, and it's easy to exhaust Munt's internal message queue (which is set to be the same size as a real MT-32) followed by the Raspberry Pi's serial port buffer itself. When the queue is exhausted, data starts to be lost, and you can get a legitimate error on the LCD. This one is a won't fix, as spamming the MIDI bus faster than the receiving device can handle is sortof abusing the system - the player software should really allow you to throttle the outgoing data or just not use this behaviour. Nearly any real MIDI synthesizer will start to complain if they receive too much data at once. That said, spamming mt32-pi has helped find 1 and 2! 😃

    Please try this build and let me know how you get on: https://github.com/dwhinham/mt32-pi/actions/runs/260047412 EDIT: Use this updated one instead: https://github.com/dwhinham/mt32-pi/actions/runs/261951874

thorr2 commented 4 years ago

Thanks! Just saw this but going to sleep for the night. I will try it tomorrow evening and let you know how it goes.

On Thu, Sep 17, 2020, 2:43 PM Dale Whinham notifications@github.com wrote:

After playing with the UM-ONE Mk II I discovered more flaws in the MIDI parser.

  1. UM-ONE sends Active Sensing packets even if the host PC does not, and I'd broken the logic that handles Active Sensing in the test build, so notes were being silenced - fixed.
  2. UM-ONE is a very clever interface, it seems to be able to not only compress MIDI data using Running Status, but may also be able to adaptively re-order messages for greater efficiency when the MIDI bandwidth is nearly saturated, allowing it to compress even more. It was sending Running Status packets for simple 2-byte messages and the parser was missing a check for this (the current release of mt32-pi only handles Running Status properly for 3-byte messages).
  3. Fast-forwarding in that MIDI Player app "too far" will completely flood the receiving device with messages, and it's easy to exhaust Munt's internal message queue (which is set to be the same size as a real MT-32) followed by the Raspberry Pi's serial port buffer itself. When the queue is exhausted, data starts to be lost, and you can get a legitimate error on the LCD. This one is a won't fix, as spamming the MIDI bus faster than the receiving device can handle is sortof abusing the system - the player software should really allow you to throttle the outgoing data or just not use this behaviour. Nearly any real MIDI synthesizer will start to complain if they receive too much data at once. That said, spamming mt32-pi has helped find 1 and 2! 😃

Please try this build and let me know how you get on: https://github.com/dwhinham/mt32-pi/actions/runs/260047412

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dwhinham/mt32-pi/issues/26#issuecomment-694515136, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVRPLWPMRMI262H7LHR3BTSGJ7ITANCNFSM4RKM26FQ .

dwhinham commented 4 years ago

I know you probably haven't had a chance to test yet, but I've done even more polishing and I've updated my previous reply with another build. I'm much happier with the state of the MIDI code now, it should be very robust.

No rush on the testing, just whenever you get some time. 😄 Cheers!

thorr2 commented 4 years ago

This is definitely the best version yet! Great job! I really appreciate all of your efforts and for purchasing the Roland UM One to help fix the remaining issues! I didn't mention it previously, but I had noticed with the official version that sometimes it would get slightly screwed up like playing the wrong voices or other minor issues with notes, but resetting everything would often sound ok the next time. So far with this version, it sounds better than before in some cases and seems to be consistent so far. I definitely approve of this version. I have another idea for a feature request so I will start a new issue for that, lol. Thanks again! Edit: it turns out the idea already exists as a feature request, so I just added my comment there. It is using sysex to switch rom versions.

dwhinham commented 4 years ago

This is definitely the best version yet! Great job! I really appreciate all of your efforts and for purchasing the Roland UM One to help fix the remaining issues!

My pleasure - I'm really glad it works well for you! 😄 Buying the Roland was pretty much the only way to debug and test this quickly because before I had no MIDI adaptors that spoke Running Status. There are probably more adaptors out there that do this, but this one has proven to be quite popular.

I didn't mention it previously, but I had noticed with the official version that sometimes it would get slightly screwed up like playing the wrong voices or other minor issues with notes, but resetting everything would often sound ok the next time.

Yes, and it all makes sense now - basically all my previous MIDI code was complete garbage. Everything would work fine provided your adaptor didn't do any Running Status stuff - v0.4.0 it seems was a partial fix, but this time I think I've covered the rest of it. I spent some time going over the MIDI spec carefully and I've tried to cover all the possible states.

So far with this version, it sounds better than before in some cases and seems to be consistent so far. I definitely approve of this version. I have another idea for a feature request so I will start a new issue for that, lol. Thanks again! Edit: it turns out the idea already exists as a feature request, so I just added my comment there. It is using sysex to switch rom versions.

Excellent! I'll do a new release shortly to get this new code out to people. I'll take a look at that request again soon, too - it's a nice idea. As always, thanks for reporting the issues and testing - you've really helped me get this project into a more robust state and my understanding of low-level MIDI has come a long way (I basically knew next to nothing before I started 😃). Enjoy using it!

thorr2 commented 4 years ago

Hi Dale, I just wanted to let you know that my Mister now works perfectly without the first few notes missing. There is a setting in the Midilink.ini file on the Mister: DELAYSYSEX=TRUE. This needs to be set to FALSE and it fixes the issue.

Also, there may be a future request to add functionality for MIDI over UDP. The Mister supports this and would eliminate the need for a MIDI interface at all. If you add networking over wifi so we can install updates (FTP would be fine), then adding something like this would be an additional feature using the networking.

Thanks!

On Sun, Sep 13, 2020 at 10:26 AM Dale Whinham notifications@github.com wrote:

Okay, there's a few things going on here!

This MIDI player is trying to be "clever" when you fast-forward. What it does is it scrubs through all of the MIDI events in the song, sending all of the parameter changes and SysEx it encounters - EXCEPT note events - until it gets to the point where you wanted to seek to. Then it resumes sending notes, and you begin to hear them again - this is presumably a feature that ensures all of the parameters are applied properly like panning and volume and nothing gets missed.

The fact that the keyboard visualisation in the player is showing the notes being "played" right after you seek is probably causing some confusion here, because in reality, they aren't. I routed this player through a MIDI debugger so I could watch the MIDI traffic and the program is clearly stripping out Note On/Offs while it's seeking. The delay you're experiencing is this particular player program filtering through the hundreds of events and sending them as quickly as it can before getting to the point you seeked to, and not an mt32-pi bug.

As for MIDI restart - I'm not getting this at while seeking around in Kings Quest V using this software, so I think we have a different unrelated issue here, possibly related to #25 https://github.com/dwhinham/mt32-pi/issues/25.

I've done a significant rewrite of the MIDI parser again, and there's a test build here: https://github.com/dwhinham/mt32-pi/actions/runs/252614148

Please try it out and let me know how it works for you!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dwhinham/mt32-pi/issues/26#issuecomment-691699575, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVRPLSLO5OCRJ3YAYKN2Z3SFT6EVANCNFSM4RKM26FQ .

dwhinham commented 4 years ago

Nice - that's good to know - thanks!

Re: UDP MIDI, that's what's implied by "Network MIDI" in the Project Status section of the README - it's coming; most likely the first network feature as it's the simplest. 🙂

I still don't have a MiSTer to test UDP MIDI with that though, though I hope to be buying one soon.

thorr2 commented 4 years ago

Very cool. You don't need all the fancy stuff to start with. Just the DE-10 Nano and maybe a wifi adapter if you need wireless. Many cores require the SDRAM, but ao486 doesn't.

On Thu, Oct 15, 2020 at 3:20 PM Dale Whinham notifications@github.com wrote:

Nice - that's good to know - thanks!

Re: UDP MIDI, that's what's implied by "Network MIDI" in the Project Status section of the README https://github.com/dwhinham/mt32-pi#%EF%B8%8F-project-status - it's coming; most likely the first network feature as it's the simplest. 🙂

I still don't have a MiSTer to test UDP MIDI with that though, though I hope to be buying one soon.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dwhinham/mt32-pi/issues/26#issuecomment-709619985, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVRPLWBABZDI5XASNEHUNTSK5YRPANCNFSM4RKM26FQ .