search

asigalov61 / Morpheus

[DEPRECIATED] Symbolic MIDI Music AI implementation
Apache License 2.0
17 stars 3 forks source link

is this the place we should look now? #2

Closed gminorcoles closed 2 years ago

gminorcoles commented 2 years ago

Hi, is this repo the one we should use for music experiments? your one from last year seemed like it was pretty mature.

do you need any help? I am a musician and a coder and I need some PyTorch stuff to work on to get back into practice.

asigalov61 commented 2 years ago

@gminorcoles Yes, Morpheus is my latest creation but I am still working on it so its not quite there yet. Otherwise, it had shown good results and everything so, I would say definitely check it out.

RE: Help. Yes, anything you can help with would be really appreciated. Do you have any ideas what would you like to contribute? Or you want me to give you some projects I have not been able to do yet?

Let me know.

Thanks.

Alex

gminorcoles commented 2 years ago

@asigalov61 Thanks for your reply. I found this project because I was looking for a deep learning project that could do a credible job of generating MIDI for my own music-making, specifically to join a challenge in algorithmic or generative music.

I have a lot of pre-GPU era machine learning experience and lots of python experience but PyTorch itself, I still dont get around as well as pandas or another ubiquitous framework. So I dont really have a laundry list, I would just take tasks from your list if you dont mind delegating them.

edit - deleted extraneous paragraphs

asigalov61 commented 2 years ago

@gminorcoles Thank you for your response and the introduction. Much appreciated.

Yes, what you are saying makes a lot of sense. I agree that the current transformer tech can't handle music properly. Most likely like you said as well it would require AGI and/or quantum computers. However, large-scale models can handle music to some degree (MuseNet would be a great example) but obviously, models like that require tons of computer power/$$$ to make so they are out of reach of small developers.

RE: Multi-modal/multi-task learning...I was able to train Morpheus for a contrastive language-music task. It seems to work fine, even without fancy stuff, like vectors or complicated transforms. You can check it out here: https://github.com/asigalov61/Morpheus/tree/main/Model/CLMP In other words, GPT( especially with RPR) shows very versatile abilities across tasks and domains. I find GPt superior to say XL or BERT because of music (only GPT can handle music properly, particularly continuations).

Now, regarding pytorch...Guys did a great job as opposed to Google's TensorFlow. So I would say that it shouldn't be hard for you to learn, especially if you have experience with pre-ML stuff like classifiers or other forecasting stuff.

I have two things I have been meaning to do for Morpheus, so tell me if you want to try to do it: 1) I have been wanting to do a complete and full state-save for my GPT2RGA implementation so that the model training can be restored in case of catastrophic failure/interruption. This was requested by users so it would be something that people would like. 2) A simplier taks...I wanted to update distributed training code (from DataParallel wrapper to DistributedDataParallel wrapper because this is what current PyTorch docs recommend).

Now, if those two tasks maybe too hard for you to start with, please tell me what else you think you can do with Morpheus specifically? Its coming out nicely so any assistance with it would be really appreciated. For example, if you think you want to play with attention mechanisms, I had a few ideas like implementing full or better yet partial sparsity so that training can be more efficient and resultative. But I am not good with AI implementations at all so I would not know where to start really. I have a few links to existing code/solutions but they would have to be looked at and integrated from scratch.

Let me know your thoughts...

Alex

asigalov61 commented 2 years ago

@gminorcoles PS. Couple more links for you to check out:

Here is my Markov chain work. Maybe it will be helpful for you to get going: https://github.com/asigalov61/Markovify-Piano

And for CLIP for Music/multi-modal stuff, here is a nice colab I found I have been meaning to adopt/rework: https://github.com/moein-shariatnia/OpenAI-CLIP

asigalov61 commented 2 years ago

I also have this: https://github.com/asigalov61/Music-Equations

gminorcoles commented 2 years ago

I need to think about the best computing environment for this. AWS is expensive, but I dont have a linux machine that will work, my ubuntu workstation died and I dont like developing on Mac or Windows. Maybe digital ocean or something. When I get the environment established I think I will start on the distributed wrapper and then try to address checkpointing if you havent already implemented it by then.

asigalov61 commented 2 years ago

@gminorcoles I use paperspace. They have very good and very diverse offerings with PyTorch/Jupyter Lab. Kaggle and Binder offer nice stuff too. Of'course there is Google Colab but it is limited.

Yes, if you can do DistributedParallel wrapper thingy, it will be great!

Thank you

asigalov61 commented 2 years ago

@gminorcoles Example of Distrubuted wrapper for RPR implementation can be found here: https://github.com/sberbank-ai/music-composer/blob/main/src/train.py

But this is for the original music transformer which was slightly modified and they use XL architecture. So I do not know if it will be compatible with my GPT.

My implementation for Morpheus is located in my other repo (I pull it from there): https://github.com/asigalov61/tegridy-tools/blob/main/tegridy-tools/GPT2RGAX.py

Hope this will help you to get going :)