Firefox mic audio input doesn't work

[ctrlaltdavid]

Firefox mic input doesn't work because it doesn't automatically handle different sampling rates in the Web Audio context that is grabbing the mic audio input and feeding it to the 24kHz Vircadia audio pipeline. Edge/Chromium, on the other hand, automatically do the conversion.

Firefox console error message: "Uncaught (in promise) DOMException: AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported."

We need to resample audio ourselves. For example, https://stackoverflow.com/questions/52787510/reducing-sample-rate-of-a-web-audio-spectrum-analyzer-using-mic-input

Probably best to implement as a new AudioWorkletProcessor alongside the existing AudioInputProcessor.ts, and insert it into the audio chain in AudioInput.#setUpAudioContext() - and perhaps insert only if Firefox or that error message is encountered.

[namark]

I've looked into this issue a bit, and I don't think a standalone AudioWorkletProcessor can be used for resampling, since the input and output are expected to be of the same sample rate. I want to try doing it AudioInputProcessor.ts before sending the data off using the approach outlined here with an OfflineAudioContext, instead of processing the arrays directly. The buffering that the worklet is doing would also help avoid resampling artifacts, although to get it absolutely perfect we might need the buffer size adjusted based on source sample rate.

[ctrlaltdavid]

The AudioContext created in AudioInput.ts probably doesn't need to be at the Vircadia sample rate - it could be at the mic sample rate. All the current AudioInputProcessor worklet does is accept incoming audio data and package it up into Vircadia-sized samples (currently assuming that the incoming data is at the required sample rate) which it posts as messages out to be sent (i.e., aren't forward into another audio node). Perhaps a resampling step could be inserted into the current audio input worklet.

[namark]

Well, hit another brick wall with my idea of using OfflineAudioContext, as it's not available to audio worklets. In fact most of everything is not available as they run in a different context. I don't want to throw it all away yet, so will try to move the processing out of the worklet. If that doesn't work out either I guess will have to use one of the hand rolled resampling algorithms.