seanmtracey
commented
4 years ago OK, here's my thinking:
In our getUserMedia constraints we can pass a desired latency target. While the stream will be unlikely to achieve the exact latency we ask for, the browser will do its best to achieve that and will report it with stream.getAudioTracks()[0].getSettings().latency.
Except sometimes it doesn't. While Chrome (on macOS, at least) will return what the latency is using the above line of code, it seems Firefox ignores the constraint completely (which this Bugzilla ticket would suggest is the case). When Chrome returns the reported latency, however, my tests show it to be more accurate than our current client-side offset_analyser.
So, here's what I suggest we implement in our code.
- We pass through the getUserMedia constraint with a latency value
- If a value is reported by the stream, we use that as our client-side offset value, and we don't run our own offset_analyser.
- If no value is returned, we run our own offset_analyser and record the value
- If no value is returned, we look to localStorage for previous analysis and the take the average of the offsets for past performances as a clue to the current latency of the stream.
- If there are no previous values recorded, we stored the offset as -1 in the database so that we have an indication that calibration totally failed.
What do you think to that?
glynnbird
The "offset" - the time correction required to align the second recorded part with the leader is pretty much a constant. Here's some experimental data:
Applying an offset of zero is pretty disastrous for the use-case where completely different parts are being recorded e.g. a garage band. The Librosa-offset calculator can't hope to help matching a piano to a drum machine
I propose that once a valid offset is recorded, it is stored in the browser's local storage we then have two choices: