Yikai-Liao / symusic

A cross platform note level midi decoding library with lightening speed, based on minimidi.
https://yikai-liao.github.io/symusic/
MIT License
108 stars 8 forks source link

Inconsistency with Score when writing to Audio #36

Open JLenzy opened 3 months ago

JLenzy commented 3 months ago

I want to do the following (in the context of a gradio interface):

  1. Take in a MIDI file, and write it out immediately to audio
  2. Process the MIDI file, (via Miditok/neural net) and save this as a new MIDI file
  3. Convert these (new) MIDI files back into symusic Score
  4. Convert these processed scores into audio as well

The new MIDI scores are not being 'allowed' by the symusic synthesizer.render function.

Original MIDI (this works fine):

s = Score(midi_file)
audio = synth.render(s, stereo=True)

print(type(s)) returns: 
<class 'symusic.core.ScoreTick'>

New MIDI (this breaks):

gen_midi.dump_midi(outpath)
s = Score(outpath)
gen_audio = synth.render(s, stero=True)

print(type(s)) returns: 
Score(ttype=Tick, tpq=480, begin=0, end=7200, tracks=1, notes=27, time_sig=1, key_sig=0, markers=0, lyrics=0)

I've verified that the new MIDI file is in fact valid. Sorry if I'm missing something obvious. Why does the second 'Score' return a different type?

Yikai-Liao commented 3 months ago

Can you provide the two midi files for subsequent testing? I will try to debug

JLenzy commented 3 months ago

Github isn't letting me upload .mid so here is a gdrive link: https://drive.google.com/drive/folders/1yZo1XlaEZU-WIW6uQkzULbCJ4M-TDAm1?usp=sharing

Yikai-Liao commented 3 months ago

Thanks for the feedback. But I would like to remind you that the audio synthesis feature, at the moment, is still in an experimental stage, so if you want to use the synthesized results for experimentation or engineering purposes, I would recommend you to use fluidsynth (the audio synthesis backend used by musescore) from the command line, which provides more stable synthesis results.

Yikai-Liao commented 3 months ago

The reason I didn't use fluidsynth directly in symusic is because fluidsynth has a lot of other dependencies that can't be installed across platforms very easily. Whereas our own rebuilt wheel can be statically linked directly into symusic without creating other dependencies.

JLenzy commented 3 months ago

Yes, your synthesizer is much easier and I'm very grateful for it! But in this case it works flawlessly with the first input. Do you have any idea why calling the Score() class on the second input MIDI file is returning a totally different type? That seems to be an issue that occurs prior to the synthesis stage

Yikai-Liao commented 3 months ago

The first one shows indeed type information, but the second one looks like repr

JLenzy commented 3 months ago

In this case, what is the solution? Am I creating the initial score incorrectly perhaps? Or is it something I can convert?

Yikai-Liao commented 3 months ago

Your code looks correct, but I don't have time to do further verification right now. I'm preparing a 3.24 gre test.

JLenzy commented 3 months ago

No worries at all =) Thanks again for all your patience and support!

Yikai-Liao commented 3 months ago

I have test the midi files, but symusic just wokrs correctly on my macbook, although I did meet a known issue mentioned in the README (strange current noise at the end of audio).

image

after.mp3.zip

Yikai-Liao commented 3 months ago

@JLenzy Can you still reproduce the problem?