How to handle audio codec/MIME

[joelle-o-world]

Audio now uploads correctly, but it comes through with the type: audio/webm;codecs=opus

Use ffmpeg to save the file correctly.

[joelle-o-world]

@sdnorris Did p5 convert it into a wav for you when you had it working before?

[sdnorris]

Yeah, p5 converted to wav by default, but if it's based on the MediaStream API there's probably a method to do that??

[joelle-o-world]

As a quick fix, I've made it so that the server will save the files in whatever format they arrive (provided that they are in our list supported file formats server/config.js for security reasons)

[joelle-o-world]

Ah ok, so the MIME can be passed to the MediaRecorder constructor, will fix now..

[joelle-o-world]

However, chrome doesn't accept audio/wav :L

[joelle-o-world]

I've had a closer look at the source code for p5.SoundRecorder

extract:

this._workletNode.port.onmessage = function(event) {
      if (event.data.name === 'buffers') {
        const buffers = [
          new Float32Array(event.data.leftBuffer),
          new Float32Array(event.data.rightBuffer)
        ];
        this._callback(buffers);
      }
    }.bind(this);

It seems like they aren't using the MediaRecorder API. Instead they set up an AudioWorkletNode to expose the pcm data and combine that into a buffer.

[joelle-o-world]

It seems that recording WAV audio using MediaRecorder is not supported by chrome. I think the best solution is still to use ffmpeg to transcode audio server side. This way the clients can use whatever is the preferred native MIME.