magenta / magenta-js

Magenta.js: Music and Art Generation with Machine Learning in the browser
https://magenta.tensorflow.org
Apache License 2.0
1.96k stars 311 forks source link

Issues with melody generation #673

Open rosmineb opened 1 year ago

rosmineb commented 1 year ago

Hello,

I'm trying to use Magenta to generate a random melody.

I copied the test case: https://github.com/magenta/magenta-js/blob/485b2ccd407f9089ef4aaceb0096f50dafe047db/music/src/music_rnn/model_test.ts#L53-L64

but I'm getting strange results, specifically I have 2 issues:

  1. When I use the temperature parameter, the output is empty
const melodyrnn = new mm.MusicRNN(
  // "https://storage.googleapis.com/magentadata/js/checkpoints/music_rnn/melody_rnn"
  "https://storage.googleapis.com/magentadata/js/checkpoints/music_rnn/basic_rnn"
);

const mel_teapot = {
  notes: [
    { pitch: 69, quantizedStartStep: 0, quantizedEndStep: 2, program: 0 },
    { pitch: 71, quantizedStartStep: 2, quantizedEndStep: 4, program: 0 },
    { pitch: 73, quantizedStartStep: 4, quantizedEndStep: 6, program: 0 },
    { pitch: 74, quantizedStartStep: 6, quantizedEndStep: 8, program: 0 },
    { pitch: 76, quantizedStartStep: 8, quantizedEndStep: 10, program: 0 },
    { pitch: 81, quantizedStartStep: 12, quantizedEndStep: 16, program: 0 },
    { pitch: 78, quantizedStartStep: 16, quantizedEndStep: 20, program: 0 },
    { pitch: 81, quantizedStartStep: 20, quantizedEndStep: 24, program: 0 },
    { pitch: 76, quantizedStartStep: 24, quantizedEndStep: 32, program: 0 },
  ],
  quantizationInfo: { stepsPerQuarter: 4 },
  totalQuantizedSteps: 32,
};

async function generateMelody() {
  try {
    let resultSeq1 = await melodyrnn.continueSequence(mel_teapot, 15, 1);
    let resultSeq2 = await melodyrnn.continueSequence(mel_teapot, 15);
    console.log("resultseq notes", resultSeq1.notes); // this is empty
    console.log("resultseq notes", resultSeq2.notes); // this is not empty
  } catch (error) {
    console.log("error", error);
  }
}
  1. Even when the output is not empty, it is very repetitive. In the example above, resultsSeq2.notes is [ Note {pitch: 52, quantizedStartStep: 4, quantizedEndStep: 7}, Note {pitch: 77, quantizedStartStep: 7, quantizedEndStep: 9}, Note {pitch: 76, quantizedStartStep: 9, quantizedEndStep: 10}, Note {pitch: 77, quantizedStartStep: 10, quantizedEndStep: 11}, Note {pitch: 77, quantizedStartStep: 11, quantizedEndStep: 12}, Note {pitch: 77, quantizedStartStep: 12, quantizedEndStep: 13}, Note {pitch: 77, quantizedStartStep: 13, quantizedEndStep: 14}, Note {pitch: 77, quantizedStartStep: 14, quantizedEndStep: 15} ]

At first there is some variety but then it just repeats pitch 77 (which isn't in the A major scale that the teapot melody uses)

What can I do to generate more unique melodies? Thanks!

Falconairio commented 11 months ago

I am having this same issue, which is annoying as examples online seem to generate melodies no problem. Would also like an answer to this, as I'm trying to use this for a uni project