search

lucidrains / routing-transformer

Fully featured implementation of Routing Transformer
MIT License
282 stars 29 forks source link

Music Routing Transformer Colab #22

Open asigalov61 opened 3 years ago

asigalov61 commented 3 years ago

Hey guys,

I had a great experience with this implementation of the Routing Transformer thanks to the efforts of @lucidrains so I wanted to share with you my creation which is based on this code/repo/implementation.

Here is the link:

https://github.com/asigalov61/Music-Transformers-Library/tree/main/Routing-Transformer

I really liked that this RT trains well and quickly. And I really enjoyed the fast generation speeds and the quality of the output.

Thank you and I hope you may find my Colab interesting and useful.

Thanks.

Alex

P.S. @lucidrains GitHub has a new feature: Discussion Boards for the repos, so I would suggest enabling it everywhere so that things like creations could be shared separately from the Issues. Just my humble suggestion. Thanks.

asigalov61 commented 3 years ago

@lucidrains Here are the results of the decent training run IMHO. Works great!

NUM_BATCHES = int(1e5) BATCH_SIZE = 8 GRADIENT_ACCUMULATE_EVERY = 4 LEARNING_RATE = 3e-4 VALIDATE_EVERY = 100 GENERATE_EVERY = 500 GENERATE_LENGTH = 512 SEQ_LEN = 4096

training: 4%|▎ | 3582/100000 [3:55:41<104:12:46, 3.89s/it]training loss: 0.12034496665000916

Generation 2048 @ 0.6-0.8 temp...

See attached MIDI samples.

Thanks.

Music-Routing-Transformer_MIDI_Output_Samples.zip