I spent several hours tinkering with this to make it work, so I figured I would share my solutions. I knew very little about NNs before I did all this, so maybe it will help someone in a similar position.
First, you may get an error when you run predict.py. It says something along the lines of data not matching. Open predict.py in your editor and navigate to line 70, where you will find this:
model.load_weights('weights.hdf5')
Change it to this:
model.load_weights('new_weights.hdf5')
If you train the model, it will kick out a new file name like "weights-improvement-01-0.4282-bigger.hdf5". You have to change the file name used by model.load_weights each time you train the model to match the new hdf5 file generated. So after you train it, and you want to use the new model you just trained, you will need to edit line 70 to look something like this:
Weirdly, after you train the model, it doesn't seem to work any longer with new_weights.hdf5. Of course, if you want to go back to the default, just re-download the directory and start from scratch.
If you try to train the model without making any changes, it will take forever. I found two solutions to this. First, adjust the data. If you go to the midi_songs folder you will find a ton of files. Get rid of most of them--leave only 10-15 that you really like. Then, open lstm.py in your editor and navigate to line 119, where you will find this:
You can also go to lines 95, 97, and 100, where you will find the following line of code repeated:
model.add(Dropout(0.3))
I reduced this from 0.3 to 0.2 on all 3 lines. I have no idea what I'm doing, so I don't know if it actually made any meaningful difference.
The big time saver was configuring tensorflow to work with my computer's GPU. This was a pain in the ass, but worth it. As of the time I'm writing this, the following does NOT work on Python 3.7, so use 3.6.
Open your command prompt as administrator and execute this:
pip uninstall tensorflow
If you hadn't installed it yet, then it will obviously tell you that. Then execute this:
pip install tensorflow-gpu
This gave me version 1.13. Then, you need to go to nVidia's website and download a couple things (you can just search them on Google). You'll have to sign up for an account and membership, but it's free and doesn't take that long. The first thing you need to install is CUDA 9.0. If you want to also spend 6 hours troubleshooting, get a different version. If you want it to just work, get version 9.0.
It will install to a directory in program files called 'nVidia GPU Computing Toolkit.' You have to include this new directory in your PATH environment variable. This is easy. If you don't know how to do it, there are a ton of guides... it takes like 10 seconds.
Once you've done that, open the nVidia GPU Computing Toolkit folder and leave it open. Then you need to get cuDNN v7.0.5. When you finally get to the page, you won't immediately see the download. Ctrl+F for the word "archive" to find the link which expands the page and will let you navigate to the download for this.
Once cuDNN is done downloading, you have to copy the contents within the cuDNN folder into the corresponding folders in your nVidia GPU Computing Toolkit folder. You can't just add the cuDNN folder to your environment variable. The file in cuDNN's "bin" folder needs to go into the nVidia GPU Computing Toolkit "bin" folder. The file in cuDNN's "include" folder needs to go into the nVidia GPU Computing Toolkit's "include" folder, and so on for the last of the three folders in the cuDNN folder. Example: https://masoncusack.github.io/blog/assets/img/postimgs/tfgpu-getting-started/bintobin.gif
I'm using a GTX860 in my laptop and went from about 1h5m per epoch to about 15 minutes per epoch (this was before I adjusted the data). Once I trimmed down the number of midi files, each epoch took about 30 seconds.
Note that there is a delicate balance between versions. Tensorflow 1.13, CUDA 9.0, cuDNN 7.0.5. If you step a single toe out of line, the errors will haunt your dreams for eternity. If for whatever reason you get a different version of Tensorflow, you can navigate to build-info.py (or something very similar to that), open it, and see what versions of these it's expecting you to use. Your debugger should give you the file path for it. Otherwise, you'll have to navigate to your python directory, then to your tensorflow directory, to find it.
As far as the results it generates, I'm impressed. I have always had a pretty good ear, and can hear harmonies and chord progressions that aren't actually there. This program comes up with stuff I never would have thought of. I'm pretty excited to compose an accompaniment for some of the melodies it generated.
I spent several hours tinkering with this to make it work, so I figured I would share my solutions. I knew very little about NNs before I did all this, so maybe it will help someone in a similar position.
First, you may get an error when you run predict.py. It says something along the lines of data not matching. Open predict.py in your editor and navigate to line 70, where you will find this:
model.load_weights('weights.hdf5')Change it to this:
model.load_weights('new_weights.hdf5')If you train the model, it will kick out a new file name like "weights-improvement-01-0.4282-bigger.hdf5". You have to change the file name used by model.load_weights each time you train the model to match the new hdf5 file generated. So after you train it, and you want to use the new model you just trained, you will need to edit line 70 to look something like this:
model.load_weights('weights-improvement-01-0.4282-bigger.hdf5')Weirdly, after you train the model, it doesn't seem to work any longer with new_weights.hdf5. Of course, if you want to go back to the default, just re-download the directory and start from scratch.
If you try to train the model without making any changes, it will take forever. I found two solutions to this. First, adjust the data. If you go to the midi_songs folder you will find a ton of files. Get rid of most of them--leave only 10-15 that you really like. Then, open lstm.py in your editor and navigate to line 119, where you will find this:
model.fit(network_input, network_output, epochs=200, batch_size=64, callbacks=callbacks_list)Change batch_size to 32. If you don't do this, the loss function will stall out and you won't get anywhere:
model.fit(network_input, network_output, epochs=200, batch_size=32, callbacks=callbacks_list)You can also go to lines 95, 97, and 100, where you will find the following line of code repeated:
model.add(Dropout(0.3))I reduced this from 0.3 to 0.2 on all 3 lines. I have no idea what I'm doing, so I don't know if it actually made any meaningful difference.
The big time saver was configuring tensorflow to work with my computer's GPU. This was a pain in the ass, but worth it. As of the time I'm writing this, the following does NOT work on Python 3.7, so use 3.6.
Open your command prompt as administrator and execute this:
pip uninstall tensorflowIf you hadn't installed it yet, then it will obviously tell you that. Then execute this:
pip install tensorflow-gpuThis gave me version 1.13. Then, you need to go to nVidia's website and download a couple things (you can just search them on Google). You'll have to sign up for an account and membership, but it's free and doesn't take that long. The first thing you need to install is CUDA 9.0. If you want to also spend 6 hours troubleshooting, get a different version. If you want it to just work, get version 9.0.
It will install to a directory in program files called 'nVidia GPU Computing Toolkit.' You have to include this new directory in your PATH environment variable. This is easy. If you don't know how to do it, there are a ton of guides... it takes like 10 seconds.
Once you've done that, open the nVidia GPU Computing Toolkit folder and leave it open. Then you need to get cuDNN v7.0.5. When you finally get to the page, you won't immediately see the download. Ctrl+F for the word "archive" to find the link which expands the page and will let you navigate to the download for this.
Once cuDNN is done downloading, you have to copy the contents within the cuDNN folder into the corresponding folders in your nVidia GPU Computing Toolkit folder. You can't just add the cuDNN folder to your environment variable. The file in cuDNN's "bin" folder needs to go into the nVidia GPU Computing Toolkit "bin" folder. The file in cuDNN's "include" folder needs to go into the nVidia GPU Computing Toolkit's "include" folder, and so on for the last of the three folders in the cuDNN folder. Example: https://masoncusack.github.io/blog/assets/img/postimgs/tfgpu-getting-started/bintobin.gif
I'm using a GTX860 in my laptop and went from about 1h5m per epoch to about 15 minutes per epoch (this was before I adjusted the data). Once I trimmed down the number of midi files, each epoch took about 30 seconds.
Note that there is a delicate balance between versions. Tensorflow 1.13, CUDA 9.0, cuDNN 7.0.5. If you step a single toe out of line, the errors will haunt your dreams for eternity. If for whatever reason you get a different version of Tensorflow, you can navigate to build-info.py (or something very similar to that), open it, and see what versions of these it's expecting you to use. Your debugger should give you the file path for it. Otherwise, you'll have to navigate to your python directory, then to your tensorflow directory, to find it.
As far as the results it generates, I'm impressed. I have always had a pretty good ear, and can hear harmonies and chord progressions that aren't actually there. This program comes up with stuff I never would have thought of. I'm pretty excited to compose an accompaniment for some of the melodies it generated.