YatingMusic / remi

"Pop Music Transformer: Beat-based Modeling and Generation of Expressive Pop Piano Compositions", ACM Multimedia 2020
GNU General Public License v3.0
553 stars 85 forks source link

Please help with REMI to MIDI coversion #16

Closed asigalov61 closed 4 years ago

asigalov61 commented 4 years ago

Hey guys,

I would really appreciate if you can help me make my REMI colab implementation of REMI encoding work.

Here is the link. https://github.com/asigalov61/DOREMI/blob/main/DOREMI.ipynb

I have no idea how to convert REMI/ REMI model output back to midi. I would really appreciate any advice/help that you can offer/suggest.

Your proposal/encoding is very interesting and a lot of people want to try it but cuz you did not provide full implementation/colab, very few of us can eval/use your work.

Please help.

Thank you so much.

asigalov61 commented 4 years ago

Never mind. I seem to solve all errors/bugs. Sorry to have bothered you.

Check out my work here: https://github.com/asigalov61/DOREMI

You are welcome to add it/link to it so that your users can use it/try it too.

Let me know if you are interested in example implementations and I will provide you with one as soon as it is ready.

I am using minGPT+your REMI code to see if SOTA results can be achieved.

Thank you again and let me know please.

remyhuang commented 4 years ago

Hi,

Thanks for your re-implementation on the colab. You can find the code segments for converting REMI to MIDI in the utils.py start from #206. If you have any other questions, please let us know.

Looking forward to the results from your re-training on minGPT!

asigalov61 commented 4 years ago

Thank you so much for your response and your advice.

I did use that code but it produces strange results. I am just trying to figure out if it is my bad coding skills or if there is a problem with your code/implementation.

I am attaching two samples for you to see because one is perfect and the other one is not so much. I am also attaching the random original sample of the input MIDIs.

It is gotta be a problem somewhere in the TXT-REMI-MIDI part because MIDI-REMI-TXT works as expected and w/o any issues.

Just in case, this is a post-processing issue/matter and AI is not involved here.

Samples.zip

remyhuang commented 4 years ago

I suspect there may be something wrong with the "position" events or "bar" events. I recommend you to check the generated events yourself before converting to MIDI.

#############################################################################################
# WRITE MIDI
#############################################################################################
def word_to_event(words, word2event):
    events = []
    for word in words:
        event_name, event_value = word2event.get(word).split('_')
        events.append(Event(event_name, None, event_value, None))
    return events

def write_midi(words, word2event, output_path, prompt_path=None):
    events = word_to_event(words, word2event)
    print(*events, sep='\n') #this line
asigalov61 commented 4 years ago

Yes, thank you again so much for your help and advice. I double-checked everything and I, unfortunately, can't pinpoint the problem without investing a considerable amount of time and resources into debug/fixes.

Don't get me wrong, it very well may be my code as I am not a particularly talented coder but unfortunately I can only use/find useful implementations/code which works and also complete as I have mentioned earlier.

So please know that for now, I won't be able to utilize your tech/code, which is very disappointing as I think that your overall proposal is fantastic and has great potential.

I am very sorry about it and I really hope that you will find a suitable purpose for REMI. And please keep me in mind/posted about REMI updates/upgrades as I will be more than happy to give it another try.

Otherwise, great job, and you definitely going in the right direction with REMI.

Feel free to contact me if needed.

Most sincerely,

Aleksandr Sigalov

P.S. GPT2 did not produce any significant output due to this issue (at least on my end) and I always manually check everything, especially the dataset/encoding, so I am more than sure now that there is a problem with the code somewhere.

P.P.S and the only significant result with REMI was that AI loved it and i.e. the consistently achieved GPT2 model training loss was very low (~0.08-0.15) which is a very good sign and result IMHO.