search

magenta / magenta-js

Magenta.js: Music and Art Generation with Machine Learning in the browser
https://magenta.tensorflow.org
Apache License 2.0
1.99k stars 314 forks source link

Add mt3 to checkpoints usable? #626

Closed rhelsing closed 2 years ago

rhelsing commented 2 years ago

Would it be possible to add the transcription checkpoint? Would it work with the OnsetsAndFrames model? I was hoping it would be reference-able like so:

model = new mm.OnsetsAndFrames('https://storage.googleapis.com/magentadata/js/checkpoints/transcription/mt3');

https://colab.research.google.com/github/magenta/mt3/blob/main/mt3/colab/music_transcription_with_transformers.ipynb

cghawthorne commented 2 years ago

Unfortunately no. The architecture differences between the two models are substantial, and we don't have a javascript implementation for MT3. One big difficulty would be getting a Transformer implementation that is performant enough in javascript. We're exploring ways to make the model easier/faster to use, but for now, I recommend just using the colab notebook.

rhelsing commented 2 years ago

@cghawthorne thanks for getting back to me!

rhelsing commented 2 years ago

@cghawthorne Would you be able to point me to the best resource for getting the model up and running on a local machine in python? as opposed to a collab notebook.

cghawthorne commented 2 years ago

The code for running locally will be basically the same as what's in the colab notebook. So I'd start by copy/pasting the colab notebook into a local python file.