search

IraKorshunova / folk-rnn

folk music modelling with LSTM
MIT License
341 stars 68 forks source link

readme

My image

Folk music style modelling using LSTMs

See the following websites:

  1. Let's Have Another Gan Ainm (music album)
  2. folkrnn.org
  3. The Machine Folk Session

Using conda, do the following:

conda create --name folk-rnn python=2.7
conda activate folk-rnn
conda install mkl-service
pip install --upgrade https://github.com/Theano/Theano/archive/master.zip
pip install --upgrade https://github.com/Lasagne/Lasagne/archive/master.zip
git clone https://github.com/IraKorshunova/folk-rnn.git
cd folk-rnn

Then to generate using one of the pretrained models:

python sample_rnn.py --terminal metadata/folkrnn_v2.pkl

To train a new model:

python train_rnn.py config5 data/data_v2

This code was used for the following published works:

  1. Sturm and Ben-Tal, "Folk the algorithms: traditional music in AI music research", The Sound of AI (Medium)

  2. Sturm and Ben-Tal, "Let’s Have Another Gan Ainm: An experimental album of Irish traditional music and computer-generated tunes"

  3. Sturm, Ben-Tal, Monaghan, Collins, Herremans, Chew, Hadjeres, Deruty and Pachet, “Machine learning research that matters for music creation: A case study,” J. New Music Research 48(1):36-55, 2018.

  4. Sturm "How Stuff Works: LSTM Model of Folk Music Transcriptions", invited presentation Joint Workshop on Machine Learning for Music, 2018.

  5. Sturm, “What do these 5,599,881 parameters mean? An analysis of a specific LSTM music transcription model, starting with the 70,281 parameters of its softmax layer,” in Proc. Music Metacreation workshop of ICCC, 2018.

  6. Sturm and Ben-Tal, "Taking the Models back to Music Practice: Evaluating Generative Transcription Models built using Deep Learning,” J. Creative Music Systems, Vol. 2, No. 1, Sep. 2017.

  7. Sturm, Santos, Ben-Tal and Korshunova, "Music transcription modelling and composition using deep learning", in Proc. 1st Conf. Computer Simulation of Musical Creativity, Huddersfield, UK, July 2016.

  8. Sturm, Santos and Korshunova, "Folk Music Style Modelling by Recurrent Neural Networks with Long Short Term Memory Units", Late-breaking demo at the 2015 Int. Symposium on Music Information Retrieval

  9. The folk-rnn v1, v2 and v3 Session Books https://highnoongmt.wordpress.com/2018/01/05/volumes-1-20-of-folk-rnn-v1-transcriptions/

  10. 47,000+ tunes at The Endless folk-rnn Traditional Music Session http://www.eecs.qmul.ac.uk/~sturm/research/RNNIrishTrad/index.html

Music compositions resulting from versions of this code:

  1. "Cloudberry Lane" by Zoë Gorman + folk-rnn (2019) https://youtu.be/6-XDhZ_AVGQ

  2. Laura Agnusdei and guest perform some output of folkrnn at the 2019 ReWire festival https://www.thoughtsource.org/vimeo_tab/app/tab/view_video?fbPageId=109823292380435&page=1&videoId=345886314

  3. "Bastard Tunes" by Oded Ben-Tal + folk-rnn (v2) (2017) https://www.youtube.com/playlist?list=PLdTpPwVfxuXpQ03F398HH463SAE0vR2X8

  4. "Safe Houses" by Úna Monaghan + folk-rnn (v2) (for concertina and tape, 2017) https://youtu.be/x6LS9MbQj7Y

  5. "Interpretations of Computer-Generated Traditional Music" by John Hughes + folk-rnn (v2) (for double bass, 2017) https://youtu.be/GmwYtNgHW4g

  6. "Dialogues with folk-rnn" by Luca Tuchet + folk-rnn (v2) (for smart mandolin, 2017) https://youtu.be/pkf3VqPieoo; at NIME 2018 https://youtu.be/VmJdLqejb-E

  7. "The Fortootuise Pollo" by Bob L. Sturm + folk-rnn (v1) (2017) https://soundcloud.com/sturmen-1/the-fortootuise-pollo-1

  8. "March to the Mainframe" by Bob L. Sturm + folk-rnn (v2) (2017) Performed by Ensemble x.y: https://youtu.be/TLzBcMvl15M?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/march-to-the-mainframe-by-bob-l-sturm-folk-rnn-v2 Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

  9. "Interlude" by Bob L. Sturm + folk-rnn (v2) (2017) Performed by Ensemble x.y: https://youtu.be/NZ08dDdYh3U?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/interlude-by-bob-l-sturm-folk-rnn-v2 (synthesized version: https://soundcloud.com/sturmen-1/interlude-synthesised) Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

  10. "The Humours of Time Pigeon" by Bob L. Sturm + folk-rnn (v1) (2017) Performed by Ensemble x.y: https://youtu.be/1xBisQK8-3E?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/the-humours-of-time-pigeon-by-bob-l-sturm-folk-rnn-v1 (synthesized version: https://soundcloud.com/sturmen-1/the-humours-time-pigeon-synthesised) Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

  11. "Chicken Bits and Bits and Bobs" by Bob L. Sturm + folk-rnn (v1) (2017, 2019) https://soundcloud.com/sturmen-1/chicken-bits-and-bits-and-bobs Score is here: https://highnoongmt.files.wordpress.com/2017/04/chicken_score.pdf

  12. "The Ranston Cassock" by Bob L. Sturm + folk-rnn (v1) (2016) https://youtu.be/JZ-47IavYAU (Version for viola and tape: https://highnoongmt.wordpress.com/2017/06/18/the-ranston-cassock-take-2/)

  13. Tunes by folk-rnn harmonised by DeepBach (2017)

    1. "The Glas Herry Comment" by folk-rnn (v1) + DeepBach (2017) https://youtu.be/y9xJl-ljOuA
    2. "The Drunken Pint" by folk-rnn (v1) + DeepBach (2017) https://youtu.be/xJyp7vBNVA0
    3. X:633 by folk-rnn (v2) + DeepBach (2017) https://youtu.be/BUIrbZS5eXc
    4. X:7153 by folk-rnn (v2) + DeepBach (2017) https://youtu.be/tdKCzAyynu4

  14. Tunes from folk-rnn v1 session volumes

    1. "The Cunning Storm" adapted and performed by Bob L. Sturm and Carla Sturm (2018) https://highnoongmt.wordpress.com/2018/01/07/the-cunning-storm-a-folk-rnn-v1-original/
    2. "The Irish Show" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/11/23/folk-rnn-v1-tune-the-irish-show/
    3. "Sean No Cottifall" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/11/11/folk-rnn-v1-tune-sean-no-cottifall/
    4. "Optoly Louden" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/07/01/optoly-louden-a-folk-rnn-original/
    5. "Bonny An Ade Nullway" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/18/bobby-an-ade-nullway-a-folk-rnn-v1-tune/
    6. "The Drunken Pint" adapted and performed by Bob L. Sturm (2017) https://youtu.be/omHhyVD3PD8; performed by EECSers (2017) https://youtu.be/0gosLln8Org
    7. "The Glas Herry Comment" adapted and performed by Bob L. Sturm (2017) https://youtu.be/QZh0WSjFFDs; performed by EECSers (2017) https://youtu.be/NiUAZBLh2t0
    8. "The Mal's Copporim" adapted and performed by Bob L. Sturm (2016) https://youtu.be/YMbWwU2JdLg; performed by EECSers (2017) https://youtu.be/HOPz71Bx714
    9. "The Castle Star" adapted and performed by Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/08/12/deep-learning-for-assisting-the-process-of-music-composition-part-2/
    10. "The Doutlace" adapted and performed by Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/08/11/deep-learning-for-assisting-the-process-of-music-composition-part-1/

  15. Tunes from the folk-rnn v2 session volumes

    1. Transcriptions 1469, 1470 & 1472 performed by Torbjorn Hultmark (2016) https://youtu.be/4kLxvJ-rXDs
    2. X:488 performed by Bob L. Sturm (2017) https://youtu.be/QWvlnOqlSes; performed by EECSers (2017) https://youtu.be/QWvlnOqlSes
    3. X:4542 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/10/06/folk-rnn-v2-tune-4542/
    4. X:2857 adapted and performed by Bob L. Sturm and Carla Sturm (2017) https://highnoongmt.wordpress.com/2017/12/02/folk-rnn-v2-tune-2857/

  16. Tunes from the folk-rnn v3 session volumes

    1. "The 2714 Polka" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/10/the-2714-polka-a-folk-rnn-original/
    2. X:1166 Performed by Weltsauerstoff (2018) https://soundcloud.com/weltsauerstoff/1166-conquest-of-brittany
    3. X:1166 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/16/folk-rnn-v3-tune-1166/, https://youtu.be/avxXRNJvUMk (2018)
    4. X:1650 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/17/folk-rnn-v3-tune-1650/
    5. X:6197 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/25/folk-rnn-v3-tune-6197/
    6. X:8589 (A Derp Deep Learning Ditty) adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/10/08/a-derp-deep-learning-ditty/

  17. "Why are you and your 6 million parameters so hard to understand?" by folk-rnn (v2) and Bob L. Sturm (2018) https://highnoongmt.wordpress.com/2018/01/21/making-sense-of-the-folk-rnn-v2-model-part-8/

  18. "Two Burner Brew No. 1" by folk-rnn (v2) and Bob L. Sturm and Carla Sturm (2018) https://soundcloud.com/sturmen-1/two-burner-brew-no-1

  19. "A Windy Canal" by folk-rnn (v2) and Bob L. Sturm (2018) https://highnoongmt.wordpress.com/2018/01/11/making-sense-of-the-folk-rnn-v2-model-part-6/

  20. "Swing Swang Swung" by folk-rnn (v2) and Bob L. Sturm (2018) https://youtu.be/Y_DBRg6SK7E (analysis here: https://highnoongmt.wordpress.com/2018/01/03/making-sense-of-the-folk-rnn-v2-model-part-5/ )

  21. "Experimental lobotomy of a deep network with subsequent stimulation (2)" by folk-rnn (v2 with lobotomy) and Bob L. Sturm, Carla Sturm (2018) https://highnoongmt.wordpress.com/2018/01/13/making-sense-of-the-folk-rnn-v2-model-part-7/

  22. "The Millennial Whoop Reel" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/08/29/millennial-whoop-with-derp-learning-a-reel/

  23. "The Millennial Whoop Jig" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/08/28/millennial-whoop-with-derp-learning/

  24. "Eight Short Outputs ..." by folk-rnn (v1) + Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/12/20/eight-short-outputs-now-on-youtube/

  25. "Carol of the Cells" by Bob L. Sturm + folk-rnn (v2) (2017) https://highnoongmt.wordpress.com/2017/12/16/carol-of-the-cells-from-the-ai-to-the-orchestra/

  26. "It came out from a pretrained net" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/12/24/taking-a-christmas-carol-toward-the-dodecaphonic-by-derp-learning/

  27. “We three layers o’ hidd’n units are” by Bob L. Sturm + folk-rnn (v2) (2015) https://highnoongmt.wordpress.com/2015/12/16/tis-the-season-for-some-deep-carols/

  28. "The March of Deep Learning" by Bob L. Sturm + folk-rnn (v1) (2015) https://highnoongmt.wordpress.com/2015/08/15/deep-learning-for-assisting-the-process-of-music-composition-part-4/

Media

  1. July 2 2019, "Deep learner generated Irish folk music", Bruce Sterling Wired
  2. April 15 2019 The Why Factor, BBC, "Separating the art from the artist"
  3. April 17 2019 Svergies Radio, Musikguiden P3, "Artist + AI = evigt liv"
  4. March 2019 Mother Jones, Clive Thompson, What Will Happen When Machines Write Songs Just as Well as Your Favorite Musician?
  5. February 19 2019 BBC Sounds, Science and Stuff, How music works
  6. February 26 2019 BBC Click, Bach vs. AI
  7. Dec. 18 2018, "How to make sweet sounding music with a hard drive", BBC Future
  8. Dec. 18 2018, "These AI startups are creating chart-worthy music with algorithms" TechWorld
  9. Nov. 25 2018 "Missing Link: Musik ohne Musiker? KI schwingt den Taktstock " Heise Online
  10. Nov. 2 2018, "Så har en artificiell intelligens skapat 100.000 folkmusiklåtar", Dagens Nyheter
  11. Sep. 2018, "AI created more than 100,000 pieces of music after analyzing Irish and English folk tunes", KTH research news
  12. Sep. 2018, "Do computers have musical intelligence?", Science Node
  13. Sep. 2018, "AI skapar 100 000 folkmusiklåtar", Musikindustrin (Sweden)
  14. May 2018, "No humans required? If computers can be taught to compose creatively, what does that mean for the future of music?", BBC Music Magazine, Alex Marshal
  15. Feb. 28 2018, "Is music about to have its first AI No.1?" https://www.bbc.co.uk/music/articles/0c3dc8f7-4853-4379-b0d5-62175d33d557
  16. Dec. 23 2017 "AI Has Been Creating Music and the Results Are...Weird" PC Mag (http://uk.pcmag.com/news/92577/ai-has-been-creating-music-and-the-results-areweird)
  17. Nov. 18, 2017 Le Tube avec Stéphane Bern et Laurence Bloch, France https://www.youtube.com/watch?v=LQQER9479Xk
  18. Sep. 25 2017, "Intelligenza artificiale crea più di 100.000 nuovi brani musicali folk" Notizie scientifiche.it
  19. June 3 2017, "An A.I. in London is Writing Its Own Music and It Sounds Heavenly" https://www.inverse.com/article/32276-folk-music-ai-folk-rnn-musician-s-best-friend
  20. June 8 2017, "Computer program created to write Irish trad tunes" http://www.irishtimes.com/business/technology/computer-program-created-to-write-irish-trad-tunes-1.3112238
  21. June 19 2017 "Folk-RNN is the Loquantur Rhythm artist of June" (providing music for phone call waits) https://zc1.campaign-view.com/ua/SharedView?od=11287eca6b3187&cno=11a2b0b20c9c037&cd=12a539b2f47976f3&m=4 (Here is a sample: https://highnoongmt.wordpress.com/2017/06/17/deep-learning-on-hold/)
  22. June 18 2017, "Real Musicians Evaluate Music Made by Artificial Intelligence" https://motherboard.vice.com/en_us/article/irish-folk-music-ai
  23. June 1 2017, "Can an AI Machine Hold Copyright Protection Over Its Work?" https://artlawjournal.com/ai-machine-copyright/
  24. May 26 2017 The Daily Mail named our project "Bot Dylan" (http://www.dailymail.co.uk/sciencetech/article-4544400/Researchers-create-computer-writes-folk-music.html), and then didn't even link to this page. Plus the video they edited has no computer-generated music in it. Well done!
  25. April 13 2017 “Eine Maschine meistert traditionelle Folk-Music” http://www.srf.ch/kultur/netzwelt/eine-maschine-meistert-traditionelle-folk-music
  26. March 31 2017, "‘Machine folk’ music composed by AI shows technology’s creative side", The Conversation (https://theconversation.com/machine-folk-music-composed-by-ai-shows-technologys-creative-side-74708)