Select audio output channels

[ivnc]

CC @Qchristensen.

Is your feature request related to a problem? Please describe.

If you have a soundcard with multiple channels (more than two), NVDA always uses 1-2 channels for its audio output, while one might prefer 3-4, 5-6 or even mono output through only one of that channels (this also includes 2-channel soundcards where the user wants NVDA only on one channel, similar to Sound Splitter add-on).

Describe the solution you'd like

Let user choose channels for NVDA audio output, through a checkbox list, for example, at Select syntesizer dialog.

Describe alternatives you've considered

Additional context

Request from a Spanish-speaking user using a quadrafonic soundcard from a DJ controller.

[cary-rowen]

This is a duplicate of #12985, I really hope NV Access adds this feature. For me @mltony's sound splitter may not be enough, I prefer @paulber19's NVDAExtensionGlobalPlugin

[ivnc]

Hmm... Not exactly, since this proposes to select audio output channels, without affecting system audio, and is intended for users who have, for instane, DJ controllers whose soundcard has speaker output on 1-2 and earphone (pre-cue) on 3-4 so they want NVDA on 3-4 (earphones).

[lionel2121]

Hola es un gusto saludarles!, Definitivamente esta función sería muy útil para los Djs que utilizamos NVDA. Espero puedan añadirla para poder usar correctamente El lector de pantalla mientras trabajamos.

[michaelDCurran]

It may be the case that the audio driver exposes these channels using only one particular audio API. We use winMM to interact with audio hardware. It would be good to find out what other applications do list these extra channels, and what audio aPI (audioSession, WASAPI, ASIO) they are using.

[lionel2121]

Hola Muchísimas gracias por el interés!, Desde hace muchos años utilizó muchísimos controladores de marcas como: Pioneer, numark, Denon, Hércules y otras más. Igualmente probado los métodos siguientes utilizando WASAPI, ASIO. Gracias espero que esté información sirva.

[ivnc]

The user mentions four DJ applications that can make use of channels 3-4: VirtualDJ, SeratoDJ, TraktorDJ and DJUsed. Don't know which API do they use.

Thanks.

[Adriani90]

cc: @jcsteh I guess implementing this now using WASAPI might be different in technical terms than before right?

[jcsteh]

WASAPI likely has an impact here, yes. I'm actually not sure how WASAPI manages these kinds of extra channels. WASAPI has an API to control the volumes of individual channels on individual streams, but I don't think that's what we want here.

CC @ahicks92, who I believe might understand this part of WASAPI better than i currently do.

[ahicks92]

Extremely brief TLDR: flag this wontfix because it is very complicated and while interesting is decidedly niche with other solutions like Virtual Audio Cable (yes yes the horror, I know). TLDR: open the device in the native channel format and zero all channels which you don't want to use, copying your mono output into the channels you do want to use. More, possibly more than @jcsteh would need, explanation follows.

(also the below is why I brought miniaudio up on the PR: you are about to learn about the horrors of audio programming in general. In fairness to Windows, other platforms are this bad as well and in fact Window's version is at least mostly documented...)

WaveOut gives you stereo unless you do a bunch of things I don't know about anymore because WaveOut is ancient. We, presumably, don't care, but as far as I can remember it's inaccurate to say that it only does stereo.

Wasapi gives you:

• The Windows mixer, which is the "default device"/the magic "automatically route"/"just handle it for me" option. You get it via I forget what exactly, but it's probably what you did now, and it's definitely what you get if you use the automatic stream routing guids I linked in the other thread. This:
  • Works in f32, for the most part.
  • Will try it's best to meet the format you ask for, e.g. channel conversions. Exactly what it will do for you is underdocumented.
  • Iirc will advertise the default format as whatever the default audio output is when you open the stream, but maybe not.
• Devices in shared mode. These generally only let you write in the device's current output channel format, though there might be some conversion. When we get to this point though you're sort of talking directly to the driver.
• Devices in exclusive mode, which are like devices in shared mode, except that the app has exclusive access (duh) and can change the device's channel formats if the device supports it. "change the device's channel format" is not a useful description because it is up to the driver to do whatever.

You can ask Wasapi about supported formats via IAudioClient. If you think this is as simple as an iterator, more fool you; you have to specifically propose formats you want or ask it for the default and deal with that. In this case, asking for the default will get you all the channels. If NVDA's current implementation is saying "give me mono" or whatever instead then technically per the docs it's broken, but in practice almost everything (everything I have personally encountered) will at least handle mono/stereo as a supported format anyway so it ends up being fine. The thing you're supposed to do is fill out WAVEFORMATEX and call IAudioClient::isFormatSupported, or call IAudioClient::getMixFormat...then do a bunch of complicated interpretation to discover just what you got back. I believe some of the newer IAudioClientN interfaces offer more useful methods.

I think what is being asked for here is this:

• Rather than opening the device with the format NVDA wants, open it with the format the device says is the default, which exposes all the channels. This may be more than 2. Possibly, you need to open it with the device's native Sr as well; if so, Cockos WDL has a resampler you can shove in that is NVDA-compatibly licensed.
• For all device types, shove NVDA's audio into into the first two channels by default. This looks like [x, x, 0, ...] repeating forever, where x is the output sample from the synth in that frame (so, basically, x, x, 0, x, x, 0 and so on if it has 3 channels). This works out because, somehow, everyone across all the platforms agree that the first two channels are an at least okay place to put stereo. You don't want to play with the others; that's a great way to end up sending NVDA's synth full power to a subwoofer.
• Now, instead of modeling that as fixed positions, expose a list of checkboxes. Each box, if checked, copies the mono stream to that channel in the frame. All the others get set to 0.

Here comes the problem. Right now, presumably you're just saying "I have mono, please deal with it". I haven't read the implementation but if you were doing something more complicated you'd not have to ask the question, so I think that's a safe assumption (I might read the implementation this weekend; there aren't many people around with the background). That is nice because while "first two channels of the default format however many channels it has always works" is indeed a kind of valid heuristic, good enough for e.g. games, I'm pretty sure it doesn't work everywhere. And as I have already said, some drivers are buggy and just flat out lie. If you go read Miniaudio you can find out just how involved that thing gets with figuring out the actual channel format and doing reasonable things with these conversions. What is being asked for here is "don't do any conversion for me", but what most users want is "sounds good"; how these are different is a whole other topic, and it isn't documented what the Windows mixer does or doesn't do before we even talk about not using it. If you open this can of worms, I honestly don't expect you'll close it anytime soon. In so much as Wasapi has training wheels on it, "give me mono/stereo f32 shared mode" is what they are and doing this will require taking them off. If the demand did exist, I would first propose just throwing your hands up and saying "fine, now you can write add-ons that replace NVWave, go forth and break stuff for the sake of your weird routing need that few other people have" and trying not to make it NVDA's problem. Something I have learned over the years is that someone somewhere always has yet another weird routing need anyway, so I highly doubt this would be the last ask.

To be clear though I'm not saying it's not doable. I personally wouldn't want to do it, though.

[jcsteh]

In case this helps anyone:

The Windows mixer, which is the "default device"/the magic "automatically route"/"just handle it for me" option.

NVDA currently uses IMMDeviceEnumerator::GetDefaultAudioEndpoint(eRender, eConsole). As I understand it, that actually gives you back the real endpoint, not some virtual auto routing endpoint. The reason I say that is that it will not switch devices automatically if the default device changes. As noted in https://github.com/nvaccess/nvda/pull/14697#issuecomment-1529178960, you can have Windows do this automatically, but that's only available in Windows 10 1607 and later.

Will try it's best to meet the format you ask for, e.g. channel conversions. Exactly what it will do for you is underdocumented.

NVDA doesn't ask what format the device supports at all. Instead, it passes the format requested by the caller, using AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM | AUDCLNT_STREAMFLAGS_SRC_DEFAULT_QUALITY to engage the Windows resampler. However, this says nothing about how it handles channel configuration mismatches.

NVDA doesn't currently do any manipulation of the raw audio data. That wouldn't be that hard to implement, but it's worth noting because any raw channel munging will require that to be implemented.

[ahicks92]

So the fun thing is that the Windows audio mixer doesn't follow devices unless you ask it to do so, I believe, and also I think it can run on multiple devices as needed too.

There's a degree of Microsoft realizing they made an unusable API and adding features in a haphazard way over the years, then not clearly documenting it in one place. Whether it's the Windows audio mixer kind of doesn't matter because you'd still end up treating it the same in this case, though. Trying to untangle that doesn't add much value to this discussion.

My guess is that back in the mists of time of 2015 or whenever they discovered that while WaveOut was deprecated, no one wanted to use Wasapi because of how terrible it used to be: no format conversions, no auto-routing, lots of crashes if you breathed on it wrong...but we will, of course, never know the actual story. Nor does it much matter to this discussion given that Vista is no longer relevant.