search

mozilla / cubeb-coreaudio-rs

The audio backend of Firefox on Mac OS X.
ISC License
25 stars 10 forks source link

Reclock audio input, when needed #153

Closed padenot closed 2 years ago

padenot commented 2 years ago

This won't compile until the changes are merged in https://github.com/mozilla/cubeb/pull/690 and https://github.com/mozilla/cubeb-rs/pull/62.

When using aggregate devices, macOS reclocks for us. When not using them, it may be that they are still in the same clock domain, in which case no reclocking is necessary. If that's not the case, we want to reclock.

padenot commented 2 years ago

The idea of the logic here seems good, but I don't have the knowledge of the new cubeb_resampler design we are going to use. Could you give me some context?

Now mozilla/cubeb#690 and mozilla/cubeb-rs#62 are already merged, we should be able to make the tests work.

Generally, if the resampler finds that there are not enough input frames compared to output frames, it will very carefully strech the input audio.

Here is a practical example. If I connect a Microsoft Lifecam VX5000 to my Macbook Pro 2018, and use it as a microphone, and then open a duplex stream with the built-in headphones for the output, with the input and output at the same rate, I receive (on average, after running a long time), 0.04% more audio input frames that there are output frames rendered.

So my new system will detect this, and adjust a resampler for the input stream by a minuscule amount so that no extra buffering happens. If the opposite hapens (input doesn't deliver enough frames), it will stretch the audio.

But with the last patches in this PR, aggregate devices always work, and macOS reclocks for us, so this doesn't happen anymore. My resampling patches will be useful for Windows though.

padenot commented 2 years ago

Can you describe a way so I can test this patch?

Find a device (A) that has both input and output channels. Examples: the Blackhole virtual device, most pro audio soundcard that have input and output jacks to connect musical instruments. If using the Blackhole virtual device you'll want to listen to the output side via something else, e.g. a DAW. I'm using either Blackhole or an old Focusrite Scarlett 6i6.

Find a device (B) that has only an input channel, such as the built-in mic of the macbook.

Set the default output device to (A), the default input device to (B), run any cubeb example that pipes the input to the output, and you should hear the correct input device in the correct output, but with an aggregate device (so reclocking is done for us internally).

padenot commented 2 years ago

Well this branch has my fix.

padenot commented 2 years ago

You know you can push to other people's PR directly, yes? It's a lot easier this way.

I've actually reverted my patch that fixes the unwrap, here is a failure: https://github.com/mozilla/cubeb-coreaudio-rs/runs/5601532591?check_suite_focus=true#step:6:526