Issue in migrating to this worklet polyfill.

TheSalarKhan commented 4 years ago

Hi, I'm facing an issue with using this polyfill. Let me first tell you why I am looking to use this. I have implemented an AudioWorket processor its working fine on chrome, I also want it to work on ios and android browsers. The issue is that the polyfill that I'm using - https://github.com/GoogleChromeLabs/audioworklet-polyfill - uses the main thread for audio rendering and that is why I am getting a glitchy audio.

Therefore I wanted to use this implementation. To explain my method of implementation. I am first going to show the default code that works without a polyfill when worklet api is present.

Default Code

Folder structure

public/
     |___ index.html
     |___ audioprocessor.js

Code

index.html

<!DOCTYPE html>
<head></head>
<body>
  <script>
    (async function () {
      // audioworker.js should also reside at root
      const audioContext = new AudioContext();

      // Initialize module and create a node for processing.
      await audioContext.audioWorklet.addModule("audioprocessor.js");
      const pn = new AudioWorkletNode(audioContext, "audioprocessor");

      // Get stream and create audio graph source
      const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
      const mic = audioContext.createMediaStreamSource(stream);
      // Connect the graph.
      mic.connect(pn);
      pn.connect(audioContext.destination);

      // Wait for 10 seconds and stop the stream as well as
      // the audio processor.
      setTimeout(() => {
        stream.getTracks().forEach(track => {
          track.stop();
        });
      }, 10000);
    })();
  </script>
</body>
</html>

audioprocessor.js

class Processor extends AudioWorkletProcessor {
    process(inputs, outputs, parameters) {
        // necessary code
    }
}

registerProcessor('audioprocessor', Processor);

Attempt at migration.

From the documentation that I came accross and from the understanding of the usage I attempted to migrate this but Its not working. Here's the details

Folder Stricture

public/
     |___ index.html
     |___ audioprocessor.js
     |___ audioworker.js
     |___ audioworklet.js

Code

index.html

<!DOCTYPE html>
<head>

<script src="audioworklet.js"></script>

</head>
<body>

    <script>
      // audioworker.js should also reside at root
      const audioContext = new AudioContext();

      // -- buflenSPN defines ScriptProcessorNode buffer length in samples
      // -- default is 512. use larger values if there are audible glitches
      AWPF.polyfill(audioContext, { buflenSPN:512 }).then(async () => {

        // Initialize module and create a node for processing.
        await audioContext.audioWorklet.addModule("audioprocessor.js");
        const pn = new AudioWorkletNode(audioContext, "audioprocessor");

        // Get stream and create audio graph source
        const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
        const mic = audioContext.createMediaStreamSource(stream);
        // Connect the graph.
        mic.connect(pn);
        pn.connect(audioContext.destination);

        // Wait for 10 seconds and stop the stream as well as
        // the audio processor.
        setTimeout(() => {
            stream.getTracks().forEach(track => {
                track.stop();
            });
        }, 10000);
      });
    </script>
</body>
</html>

audioprocessor.js

class Processor extends AudioWorkletProcessor {
    process(inputs, outputs, parameters) {
        // necessary code
    }
}

registerProcessor('audioprocessor', Processor);

I'd be grateful if someone can help me in filling the gaps. I think this is going to have much better performance on mobile browsers.

jariseon commented 4 years ago

Hi, try changing mic.connect(pn) to mic.connect(pn.input ? pn.input : pn)

the polyfilled AudioWorkletNode is not a true AudioNode, so without .input the connect() method will fail. i've an idea how to make that ugly workaround unnecessary in future but will need some time to test it.

let me know if the above workaround did not fix the issue.

TheSalarKhan commented 4 years ago

Hi @jariseon I was able to get this working. And instantly its has much better performance.

Although there were a few road blocks that I came across. I will be sharing them here and I'll try to patch these issues in my fork and send a PR.

Thanks allot for this awesome library. :+1:

TheSalarKhan commented 4 years ago

Here's the PR @jariseon

https://github.com/jariseon/audioworklet-polyfill/pull/7

For me this issue has been resolved. I'm keeping this open in case you want any discussion, else you can close it.

This performs really well man. Thanks :+1:

jariseon / audioworklet-polyfill