twilio / twilio-video-app-react

A collaboration application built with the twilio-video.js SDK and React.js
Apache License 2.0
1.81k stars 728 forks source link

Using WebAudio - AudioWorkletProcessors to process Audio Data #819

Open arvind-iyer-2001 opened 1 year ago

arvind-iyer-2001 commented 1 year ago

Question I'm tryin to use WebAudioAPIs to add my custom audio processor, (for now it just outputs the input value, I'll need it later to perform some Audio Analysis, not needed to modify the output). I have been trying to use the below code to process the audioTrack data but have not been able to see any results of its working. How can I make this work.

AddAudioProcessor Callback image

const addAudioProcessor = useCallback(async () => { if (audioTrack) { const mediaStream = new MediaStream(); mediaStream.addTrack(audioTrack.mediaStreamTrack); const audioContext = new AudioContext(); audioContext.createMediaStreamSource(mediaStream); await audioContext.audioWorklet.addModule("Test-Processor.js"); const testNode= new AudioWorkletNode( audioContext, 'test-processor' ); testNode.connect(audioContext.destination); } }, [audioTrack])

Test-Processor.js image

I have been refering to the WebAudio Documentation to implement this. https://developer.mozilla.org/en-US/docs/Web/API/AudioWorkletProcessor