search

ghalestrilo / ornette

An OSC server to interact with different music-generating ML models
12 stars 0 forks source link

readme

Ornette

Let AI Make Music

Ornette an OSC-based command-line application to use ML models for continuous music generation. You can control (manually or programatically) ML-based music generation and playback via text commands, OSC messaging or with command-line arguments. You can also try out different models through the same interface.

Ornette Demo

The preview above shows Ornette running the PerformanceRNN model in the left panel and sending data to the SuperCollider instance to the right, which then generates sound (except when you record your screen as an SVG file 🤷🏻‍♂)

Concept

Ornette is an interactive container-based music generation Machine Learning model host and MIDI data workstation. Audio playback is completely delegated to external systems: at the moment, SuperCollider using the SuperDirt quark.

Ornette delegates audio synthesis to SuperCollider and uses Docker for environment management.

Ornette allows you to:

Requirements

Running

  1. Clone the repo (git clone git://github.com/ghalestrilo/ornette.git)
  2. Run pip install -e requirements.txt
  3. Run the host with python .. You'll be prompted to choose an RNN model and bundle
    • Optionally. pass --modelname=<desired model> --checkpoint=<desired checkpoint> to dismiss these prompts
  4. To hear playback, make sure SuperCollider is running SuperDirt (either sclang or scide is fine)

Example

Use the following command to start running MelodyRNN:

python . --model=melody_rnn --checkpoint=basic_rnn

Once started, you can issue commands to the serve, such as:

Module Info

Modules are listed on the modules folder, and their bundles can be found on the .ornette.yml file. Most of the models were developed by Magenta Research. These are the currently implemented modules:

modelname checkpoint
melody_rnn basic_rnn
melody_rnn mono_rnn
melody_rnn lookback_rnn
melody_rnn attention_rnn
performance_rnn polyphony_rnn
performance_rnn performance
performance_rnn performance_with_dynamics
performance_rnn performance_with_dynamics_and_modulo_encoding
performance_rnn pitch_conditioned_performance_with_dynamics
performance_rnn multiconditioned_performance_with_dynamics
pianoroll_rnn_nade rnn-nade_attn
polyphony_rnn polyphony_rnn

Roadmap