Code for transferring timbre in browser using ddsp.synthesize? [Docs]

Pulled my hair out over this one, so leaving it here for anyone else who needs it:

                <IonButton onClick={async () => {
                  const Magenta = (await import('@magenta/music'))
                  const { DDSP, SPICE } = Magenta
                  const spice = new SPICE()
                  await spice.initialize()

                  // const audioUrl = 'https://s3-us-west-2.amazonaws.com/s.cdpn.io/123941/Yodel_Sound_Effect.mp3'; // yodel

                  const context = new AudioContext();

                  window.fetch(audioUrl)
                    .then(async response => {
                      const arrayBuffer = await response.arrayBuffer()
                      return arrayBuffer
                    })
                    .then(async arrayBuffer => {
                      const audioBuffer = await context.decodeAudioData(arrayBuffer)
                      return audioBuffer
                    })
                    .then(async audioBuffer => {
                      const audioFeatures = await spice.getAudioFeatures(audioBuffer)
                      const checkpointUrl = 'https://storage.googleapis.com/magentadata/js/checkpoints/ddsp/violin'
                      const ddsp = new DDSP(checkpointUrl)
                      await ddsp.initialize()
                      const synthesizedBuffer = await ddsp.synthesize(audioFeatures)

                      // Copied from ~/node_modules/@magenta/music/esm/ddsp/buffer_utils.js, which is
                      // called by the addReverb method in node_modules/@magenta/music/esm/ddsp/add_reverb.js,
                      // which is called by the .synthesize method in node_modules/@magenta/music/esm/ddsp/model.js
                      const arrayBufferToAudioBuffer = (audioCtx, arrayBuffer, sampleRate) => {
                        const newBuffer = audioCtx.createBuffer(1, arrayBuffer.length, sampleRate);
                        newBuffer.copyToChannel(arrayBuffer, 0);
                        return newBuffer;
                      };
                      console.log('synthesizedBuffer:', synthesizedBuffer)

                      const synthesizedAudioBuffer = arrayBufferToAudioBuffer(context, synthesizedBuffer, 48000)

                      function play(myBuffer) {
                        const source = context.createBufferSource();
                        source.buffer = myBuffer;
                        source.connect(context.destination);
                        source.start();
                      }

                      play(synthesizedAudioBuffer)
                    });
                }}>
                  <IonIcon icon={volumeHigh} />
                </IonButton>

Checkpoint info (for violin, flute, etc.) can be found here:

https://github.com/magenta/magenta-js/tree/master/music/checkpoints#table

In my experience, the checkpoint downloader script didn't really work. It also appears there are 2 generations of DDSP models: one that powers this site here...

https://magenta.github.io/magenta-js/music/demos/ddsp_tone_transfer.html

...and then one that powers the Google Colab they link to on the front page of the DDSP README:

https://colab.research.google.com/github/magenta/ddsp/blob/main/ddsp/colab/demos/timbre_transfer.ipynb

I've only been able to access the gen 1 checkpoints so far. Haven't really tried with gen 2, but my plan for that one is to run the Google Colab, and manually download the checkpoint files from disk on the left, and then host them at S3. That will also need to host the .pkl and .gin files too, AFAIK.

magenta / magenta-js

Code for transferring timbre in browser using ddsp.synthesize? [Docs] #659