Using Deep Learning to generate pop music!
You can also experiment through the web app - musicautobot.com
Recent advances in NLP have produced amazing results in generating text. Transformer architecture is a big reason behind this.
This project aims to leverage these powerful language models and apply them to music. It's built on top of the fast.ai library
MusicTransformer - This basic model uses Transformer-XL to take a sequence of music notes and predict the next note.
MultitaskTransformer - Built on top of MusicTransformer, this model is trained on multiple tasks.
Training on multiple tasks means we can generate some really cool predictions (Check out this Notebook):
Details are explained in this 4 part series:
Play with predictions on Google Colab
MusicTransformer
MultitaskTransformer
Data Encoding
Pretrained models are available as MusicTransformer and MultitaskTransformer (small and large).
Each model has an additional keyC
version. keyC
means that the model has been trained solely on music transposed to the key of C (all white keys). These models produce better results, but expects the input to all be in the key of C.
MultitaskTransformer
For details on how to load these models, follow the Generate and Multitask Generate notebooks
CLI scripts for training models:
run_multitask.py - multitask training
python run_multitask.py --epochs 14 --save multitask_model --batch_size=16 --bptt=512 --lamb --data_parallel --lr 1e-4
run_music_transformer.py - music model training
python run_music_transformer.py --epochs 14 --save music_model --batch_size=16 --bptt=512 --lr 1e-4
run_ddp.sh - Helper method to train with mulitple GPUs (DistributedDataParallel). Only works with run_music_transformer.py
SCRIPT=run_multitask.py bash run_ddp.sh --epochs 14 --save music_model --batch_size=16 --bptt=512 --lr 1e-4
Commands must be run inside the scripts/
folder
Install anaconda: https://www.anaconda.com/distribution/
Run:
git clone https://github.com/bearpelican/musicautobot.git
cd musicautobot
conda env update -f environment.yml
source activate musicautobot
Install Musescore - to view sheet music within a jupyter notebook
Ubuntu:
sudo apt-get install musescore
MacOS - download
Installation:
cd serve
conda env update -f environment.yml
You need to setup an s3 bucket to save your predictions. After you've created a bucket, update the config api/api.cfg with the new bucket name.
Development:
python run.py
Production:
gunicorn -b 127.0.0.1:5000 run_guni:app --timeout 180 --workers 8
Unfortunately I cannot provide the dataset used for training the model.
Here's some suggestions:
This project is built on top of fast.ai's deep learning library and music21's incredible musicology library.