chcunningham / wc-talk

MIT License
44 stars 5 forks source link

Bug in AudioEncoder/EncodedAudioChunk implementation #9

Closed guest271314 closed 2 years ago

guest271314 commented 2 years ago

AudioEncoder output emits 1st EncodedAudioChunk byteLength 8 and duration 60000. 2d EncodedAudioChunk emits byteLength of 409 with duration 60000 (44100 input sample rate).

That does not appear to be correct. How can an 8 byteLength EncodedAudioChunk have same duration as a 409 byteLength EncodedAudioChunk?

When Media Source Entensions is used for playback that winds up losing 1 second of playback, which is 1:35 when playing the WAV file, HTMLMediaElement currentTime reaches 1:34.

Resampling input to 48000 does not change the 1st EncodedAudioChunk byteLength nor playback ending 1 second prematurely.

<!DOCTYPE html>
<html>
  <head></head>
  <body>
    <input type="file" accept=".wav" />
    <script type="module">

      document.querySelector('input[type=file]').onchange = async (e) => {
        const data = new Int16Array(
          await e.target.files[0].arrayBuffer()
        ).slice(44);

        let config = {
            numberOfChannels: 2,
            sampleRate: 44100,
            codec: 'opus',

          };

        const encoder = new AudioEncoder({
          error(e) {
            console.log(e);
          },
          output: async (chunk, metadata) => {
            if (metadata.decoderConfig) {
              config.description = metadata.decoderConfig.description;
            }
            chunks.push(chunk);
          },
        });
        console.log(await AudioEncoder.isConfigSupported(config));
        encoder.configure(config);
        await encoder.flush();

        const audio = new Audio();
        audio.controls = audio.autoplay = true;
        const events = [
          'loadedmetadata',
          'loadeddata',
          'canplay',
          'canplaythrough',
          'play',
          'playing',
          'pause',
          'waiting',
          'progress',
          'seeking',
          'seeked',
          'ended',
          'stalled',
          'timeupdate',
        ];
        for (const event of events) {
          audio.addEventListener(event, async (e) => {
            if (e.type === 'timeupdate') {
              if (!ms.activeSourceBuffers[0].updating) {
                ms.activeSourceBuffers[0].timestampOffset = audio.currentTime;
              }
            }
          });
        }
        document.body.appendChild(audio);

        const ms = new MediaSource();
        ms.addEventListener('sourceopen', async (e) => {
          console.log(e.type, config);
          URL.revokeObjectURL(audio.src);
          const sourceBuffer = ms.addSourceBuffer({
            audioConfig: config,
          });

          sourceBuffer.onupdate = (e) => console.log(e.type);

          sourceBuffer.mode = 'sequence';
          for (const chunk of chunks) {
            await sourceBuffer.appendEncodedChunks(chunk);
          }
        });
        audio.src = URL.createObjectURL(ms);
      };
    </script>
  </body>
</html>

Screenshot_2022-07-30_17-53-48

Screenshot_2022-07-30_18-00-48

blade_runner.wav.zip