Outputting MIDIs to WAV/MP3 possible?

[sirbaratusii]

I don't know if it's exactly out of scope but if possible, I think a feature to take MIDIs and output them to WAV files would be nice, since I tend to put together a bunch of converted MIDIs for livestreams and videos.

[Karmeck]

What is the gins from doing this, instead of using the emulator with audacity, and record the midi inside audacity.

This can be done using loopmidi.

[sirbaratusii]

What is the gins from doing this, instead of using the emulator with audacity, and record the midi inside audacity.

This can be done using loopmidi.

My disadvantage of going with this route is that it could last hours if not days. whereas a feature to convert the MIDIs to WAV/MP3 can be faster. For example I have 175 MIDIs from a series of games developed by Kraisoft Entertainment. Given the total hours according to Foobar2000 I could be sitting for nearly 6 hours.

[zaphod77]

Yeah the op wants to instead of render in realtime and stream, to render at full cpu speed and write to disk.

would be an interesting feature, but not sure it's practical.

[NRS-NewRisingSun]

Seconding the request. MUNT has a similar feature with its mt32-smf2wav utility, and I use it to quickly produce .wav files that can be listened-to on any device.

[zulc22]

What is the gins from doing this, instead of using the emulator with audacity, and record the midi inside audacity.

This can be done using loopmidi.

Systems which cannot run the emulator at full speed can generate a perfect .WAV log, and systems that can run the emulator at full speed can generate a .WAV log faster than real time.

Requiring the use of:

• A MIDI player
• LoopMIDI
• Either a physical loopback, soundcard loopback, or VB-Audio Cable
• An audio recorder (in real time)

Causes there to be no extra benefit to emulating the SC-55 other than convenience, let alone all those steps adding a whole lot of cracks in which lag + latency can shove themselves into, making it easy to say the quality of the output can't be 100% trusted.

MIDI playback and WAV logging implemented inside the emulator itself would produce as perfect as an output as the code written is capable of. Not having these features could lead to lost output quality potential.

[jcmoyer]

I've prototyped this feature here https://github.com/jcmoyer/Nuked-SC55/tree/renderer (renderer branch) - needs some cleanup but MVP etc etc. Currently assumes you're running on a little-endian architecture like x86. I marked the places I think are problematic with TODO comments.

The executable produced with this fork can be run with nuked-sc55 -o output.wav input.mid to render input.mid to output.wav. It is not a drop-in replacement for the binary built from this repository. Notably, it does not accept midi input or output to an audio device. The latter had to be disabled because normally the MCU emu waits for the audio callback to advance the read pointer which is undesirable since we want to go as fast as possible.

There may be some timing issues (particularly related to tempo, but probably also emulator synchronization) that need to be ironed out. I went off of a comment in the source code that guesses one instruction is 12 cycles. 24000000 cycles = 1 second, 24 cycles = 1 microsecond, so you get 2 instructions per microsecond. I haven't tested any midi files with tempo changes mid-track, and I assume they won't play correctly without some more work.

The gist is:

1. Load MIDI events from file, merge all the tracks so there's only one list of events
2. Run the emulator until the next MIDI event should happen
3. Inject the event
4. Repeat from step 2 until the last event (hopefully MIDI end of track)

Here are some benchmarks from the Alien Vendetta MIDI Pack running on a stock i5-2500k

Song	Playback time	Render time
MAP03 - The Berggren-Malde Complex	3m44s	2m30s
MAP06 - Shoot the Pianist	2m46s	1m49s
MAP24 - Arcadia	5m33s	3m47s
MAP31 - Kaleidoscopic Array	3m12s	2m12s

It's fine if there's no interest supporting this upstream. I just wanted to see how difficult it would be to implement.

[datajake1999]

I just tested this with a MIDI that has a tempo change, and I am posting the MIDI in question as well as the resulting output converted to OGG. The tempo change causes the output to hang for a bit, and it continues normally some time later. Since the file I tested doesn't have a GM reset, the pitch is a little off as well. kickbutg.zip

[sirbaratusii]

I've prototyped this feature here https://github.com/jcmoyer/Nuked-SC55/tree/renderer (renderer branch) - needs some cleanup but MVP etc etc. Currently assumes you're running on a little-endian architecture like x86. I marked the places I think are problematic with TODO comments.

The executable produced with this fork can be run with nuked-sc55 -o output.wav input.mid to render input.mid to output.wav. It is not a drop-in replacement for the binary built from this repository. Notably, it does not accept midi input or output to an audio device. The latter had to be disabled because normally the MCU emu waits for the audio callback to advance the read pointer which is undesirable since we want to go as fast as possible.

There may be some timing issues (particularly related to tempo, but probably also emulator synchronization) that need to be ironed out. I went off of a comment in the source code that guesses one instruction is 12 cycles. 24000000 cycles = 1 second, 24 cycles = 1 microsecond, so you get 2 instructions per microsecond. I haven't tested any midi files with tempo changes mid-track, and I assume they won't play correctly without some more work.

The gist is:

1. Load MIDI events from file, merge all the tracks so there's only one list of events
2. Run the emulator until the next MIDI event should happen
3. Inject the event
4. Repeat from step 2 until the last event (hopefully MIDI end of track)

Here are some benchmarks from the Alien Vendetta MIDI Pack running on a stock i5-2500k

Song Playback time Render time MAP03 - The Berggren-Malde Complex 3m44s 2m30s MAP06 - Shoot the Pianist 2m46s 1m49s MAP24 - Arcadia 5m33s 3m47s MAP31 - Kaleidoscopic Array 3m12s 2m12s It's fine if there's no interest supporting this upstream. I just wanted to see how difficult it would be to implement.

This is very nice! Thanks for implementing this. ^^

Though just recently somebody was able to write a command-line that auto resets to GM or GS mode. Hopefully adding that in potentially could solve the pitch issues?

[jcmoyer]

I pushed a fix that should handle tempo changes properly and also rebased so you can pass -gm and -gs. Resets seem to take a while to complete and bleed into the render process if it is started immediately. The workaround for now is to just run the emulator for a couple seconds after sending the reset message but before starting the render. Let me know if you encounter any more obvious problems.

[sirbaratusii]

I pushed a fix that should handle tempo changes properly and also rebased so you can pass -gm and -gs. Resets seem to take a while to complete and bleed into the render process if it is started immediately. The workaround for now is to just run the emulator for a couple seconds after sending the reset message but before starting the render. Let me know if you encounter any more obvious problems.

Thanks! I also have another bug where some instruments don't play properly atleast when compared to the recording. More noticable at the near end of the track. I've included two OGG files (one is the recording done with Falcosoft's MIDI Player and the other one rendered. I also included the source MIDI.

endmusic8bug.zip

UPDATE : I found another bug where some MIDIs were playing faster than they should.

The result was a minute and 36 seconds when it should be 2 minutes.

OGG & Source MIDI also included EXEC_D_THE_DA_bug.zip

[jcmoyer]

I also have another bug where some instruments don't play properly atleast when compared to the recording. More noticable at the near end of the track.

So I'm unsure about this problem. If I play that midi through Sekaiju->loopMIDI->Nuked-SC55 (from this repository) notes cut out the exact same way as the wave renderer does. I get the same thing with the Falcosoft player. This strikes me as a polyphony limit somewhere in the emulator. Out of curiosity how did you record the version that doesn't have notes cutting out? Is it using a SC55 soundfont or does it go through Nuked-SC55?

I found another bug where some MIDIs were playing faster than they should.

This one is because the midi doesn't set a tempo and I had copied an incorrect starting tempo from some midi processing code I wrote a while back. It should be fixed now.

[sirbaratusii]

So I'm unsure about this problem. If I play that midi through Sekaiju->loopMIDI->Nuked-SC55 (from this repository) notes cut out the exact same way as the wave renderer does. I get the same thing with the Falcosoft player. This strikes me as a polyphony limit somewhere in the emulator. Out of curiosity how did you record the version that doesn't have notes cutting out? Is it using a SC55 soundfont or does it go through Nuked-SC55?

I used two instances of Nuked-SC55 (one for even channels and the other for odd channels) and Falcosoft's MIDI Player to get the recording done.

I am curious if there is a way to implement a command line to increase the polyphony limit? or maybe have it so it can render each channel and merge them together? (I think that could increase the render time I assume)

[jcmoyer]

I am curious if there is a way to implement a command line to increase the polyphony limit? or maybe have it so it can render each channel and merge them together? (I think that could increase the render time I assume)

It's technically doable, but I'm reluctant to make the necessary changes while this repository is in flux. All of the global state would need to be removed to support multiple emulators in one process, and there's a lot of it. It's too much code to review whenever big changes get merged upstream. That's not to say that I won't attempt it, but I'd like to wait a bit to see how much more development happens on this project.

I think the easiest way to solve this problem in the meantime would be to write a script that splits a midi file into multiple midi files, each one containing a subset of the tracks in the original midi file. Run the renderer on each of the new midi files, and finally use another script to merge the resulting wav files.

[jcmoyer]

Alright I ported my renderer to the multi-instance branch from #68 while doing some other clean up. Now you should be able to render midis to wavs with a higher effective polyphony. It uses the same strategy described in that PR. You can find the new renderer branch here. This one will build both the standard frontend and renderer frontend. I'm playing with new ideas on this branch and I'll try to keep it reasonably up to date for now.

[Kappa971]

Will we ever see all the improvements from https://github.com/nukeykt/Nuked-SC55/pull/68 in the main branch? 😁