Preliminary MidiPlayer implementation

ghost commented 3 years ago

Preliminary MidiPlayer implementation.

This is a complete impementation that also works in the export. Requires some more work integrating it but I am unfamiliar with Goost file structure. I did my best. I can fix other smaller issues in later PRs but at this point is fully functional, imports and plays in the exported game too.

This PR adds two classes

MidiPlayer: Adds the ability to play .mid or .midi files or to manually play midi notes (to create your own instrument see below for more info). A soundfont + AudioStreamGenerator must exist for sound to play (see below).
MidiFIle: A generic imported midi container that has a PoolByteArray data which contains all the binary data. The data can be accessed in GDSCript but you can only get not set (you're not supposed to manipulate binary data this way). It loads any .mid .midi and .sf2 files. There is a built in Format enum but hasn't been implemented yet so all all the same type (for now). I chose to not split it into two different resources and instead use one generic type since both had the exact same structure with data which contains binary data.

MidiPlayer has fully documented xml file but MidiFile does not at this time.

Playing the audio

There is no Inspector integration at this time. You need a script and use load_soundfont("res://soundfontname.sf2") and load_midi("midisong.mid") in any order in _ready(). You also need an AudioStreamGenerator and the entire MidiPlayer set to Playing or Autoplay to true.

The data is imported as a .mdf (MidiFile) format internally (see in .import) and is going to be saved in the pck. The MidiPlayer reads this data using ResourceLoader::load rather than FileAccess (which only works in the editor but not in the exported file). The data is then saved in memory and transfered to TinySoundFont player which does all the heavy lifting . Then finally there's an internal process buffer feeding mechanism that works automatically while there's a soundfont and an AudioStreamGenerator and MidiPlayer is playing.

Integration needs more work (IMPORTANT)

Unfortunately I am unfamiliar with Goost file structure so it probably needs more work integrating it properly. I tried my best.

You would also need to add the license . This is the thirdparty library used TinySoundFont.

Forgot to mention. credit need to be given to RodZill4 as well as I based my module implementation on his project https://github.com/RodZill4/godot-music though mine is heavily modified.

Creating your own piano

Totally possible to do but requires manually creating every single key then calling some function with note_on(instrument, note, volume). Instrument type is the built in SoundFont instrument (piano is 0), note is the actual note from 0 to 127 (60 is middle C4), volume 1.0 is max. This requires advanced understanding of how MIDI works in order to get it right but you can produce a fully functional Midi Keyboard and you can even send MIDI signals.

Note that you will get a big lag with default parameters. You need to lower the buffer size to something like 0.05 in order to get faster response so it plays right away. It really depends on the computer's ability to process audio buffers. Values below 0.04 for me crackle audio.

Some other thoughts

I opted for the name Midi (with capital M and everything else lowercase) to differentiate from MIDI events. The corret term is MIDI but Midi is also acceptable and in fact in this case preferable as it removes any confusion.

ghost commented 3 years ago

Note that checks may fail because it might not be properly integrated into Goost. I tried my best :) It fails to recognize audio component.

ghost commented 3 years ago

It won't work unless you uncomment and implement set_data otherwise data becomes disabled and it thinks is size 0. Very strange behavior. With both set_data and get_data it will work.

Xrayez commented 3 years ago

Requires some more work integrating it but I am unfamiliar with Goost file structure. I did my best.

For the future, you can refer to https://goost.readthedocs.io/en/latest/development/adding_new_features.html

For some reason, I couldn't get to play anything, even with you suggested changes.

But I know it should work, so lets merge this so I can push changes to integrate this feature.

Thanks!

goostengine / goost