[Question] What's the current status of MIDI playback or SoundFont2 support?

[BLumia]

Hi, first of all thanks for the tiny but great library!

I've already used this library in one of my side projects. My intended usage is to play simple MIDI files that vgmtrans extracted and it works great so far. Then I poked around and tried to load some MIDI files which are more complex, and noticed some noticeable differences between TinySoundFont, FluidSynth and Timidity++.

I'll attach the MIDI file for testing below, my question is: What's the current status of MIDI playback / SoundFont2 support? I saw an Issue thread (https://github.com/schellingb/TinySoundFont/issues/14#issuecomment-371381296) which is related to this question, but I'm not sure if it's up-to-date.

The midi file is: striving.zip, which is downloaded from here, the SoundFont that I used to test this MIDI file is Reality_GMGS_falcomod.sf2 which can also be downloaded from here.

Thanks again!

[schellingb]

Hey there! The comment you have linked is still up-to-date. The linked soundfont uses both mentioned generators ChorusEffectsSend and ReverbEffectsSend which are not yet implemented. It also defines modulators, but I can't easily tell if the modulators are just the default ones (which TSF adheres to) or if they are customized. This can make the generated output sound wrong by a little or a lot, depending on how much the soundfont relies on these features.

The generators can be added to the library if we have an effect implementation that makes sense for us (not too much code, not too much memory required, not too much processing required, no license limitations).

But modulators? I think it would need a large rewrite to make every generator be controllable by modulators. It would no doubt increase the performance requirements which could make it more problematic to be used for lower end platforms (weaker ARM devices or embedded platforms). I wouldn't even know where to start implementing modulators. As much as I would love to have support for them :-)

[BLumia]

Thanks for the quick response!

The comment you have linked is still up-to-date.

Just want to confirm, are the "MIDI playback" part also up-to-date? i.e.

MIDI playback: Besides regular instrument selection, note on and off, we do have support for pitchwheel, pan and volume messages. Which are the most common ones. I think the biggest missing thing is support for switches like sustain, portamento, sostenuto, soft pedal and legato. I have not evaluated how hard implementation of these would be.

While trying to test the result, I am using QMidiPlayer which is FluidSynth-based, and it allow user manually turn the Chorus and Reverb effect off. It still have noticeable difference when using the same SoundFont and Chorus and Reverb are both turned off.

For convenience, here is a rendered clip so you can hear the difference (the attachment is actually a m4a file, but GitHub won't allow me to upload m4a file directly, so I change the file suffix to mp4 instead):

https://github.com/schellingb/TinySoundFont/assets/10095765/96acc3eb-e329-4b1b-bfec-715361c4f233

The first half is rendered by the FluidSynth-based QMidiPlayer (with Chorus and Reverb disabled), the second half is rendered by my TinySoundFont and TinyMidiLoader-based MIDI player. It sounds like sustain, soft pedal and/or legato is missing in the later one, but I'm not quite sure about it.

[schellingb]

Loading the MIDI in https://signal.vercel.app/edit does show usage of the "Hold Pedal" controller which I think according to https://anotherproducer.com/online-tools-for-musicians/midi-cc-list/ is MIDI CC 64 which in TinyMidiLoader is TML_SUSTAIN_SWITCH. TML reads the control messages fine, it's just that when passed to tsf_channel_midi_control it won't do anything with unsupported controllers. All supported there is 0, 6, 7, 10, 11, 32, 38, 39, 42, 43, 98, 99, 100, 101, 120, 121, 123. This hasn't changed for many years so yeah, none of these pedal controllers are handled. Some sound fairly easy to implement though, certainly easier than adding support for chorus, reverb or modulators :-)

[BLumia]

Oh, it seems indeed caused by CC 64. Now I have a good starting point to know where to look at :)

Thanks for the info and thanks again for the great library! I'll close this issue now :D

[BLumia]

Would like to link it here so it's easier to find: https://github.com/schellingb/TinySoundFont/pull/88 this PR works quite well to address the CC64 issue mentioned above :)

[mrbumpy409]

It would also seem that MIDI CC1 is not supported in TinySoundFont, as it should by default activate the vibrato LFO.