Surround sound support? Part 2: Current state of positional audio (5.1 / 7.1) in Proton

[mrmatteastwood]

Feature Request

This is a follow-up to https://github.com/ValveSoftware/Proton/issues/1803, where intrepid users and devs like @ColdPie1, @Loacoon1 and @pedrofleck investigated surround sound support in Proton and made a bunch of headway. Unfortunately, I still run into games with broken surround sound with worrying regularity.

Setup/background:

Hooking up a home theatre system to a PC is incredibly easy. Simply plug an HDMI cable into the graphics card and AVR. I can confirm that surround sound has worked perfectly for me on Linux Mint, Ubuntu and Windows since 2009. I listen to multichannel mixes of music (e.g., Pink Floyd), I watch movies and I play video games. VLC, SMPlayer, Parole Media Player, GZDoom, Kodi, you name them, disc formats like DVD, Blu-Ray, audio formats like PCM audio, DTS, AC3, Dolby True HD, DTS-HD MA, multichannel file formats like .flac, .dts, .mka, .ac3, .mkv, .avi, .wav, .ogg... They all work. So, system-side, everything is set up correctly:

Screen 1: PC => DisplayPort cable > Screen (2560x1440) Screen 2: PC => DVI cable > Screen (1280x1024) Screen 3: PC => HDMI cable > AVR > HDMI cable > Screen (1920x1080) I run games on screen 1.

I can also confirm that this works with AVRs made by Yamaha, Denon, Panasonic, JBL and several more at my friends' places, where I bring my laptop for gaming or movie nights. Both my laptop and desktop PC have Nvidia graphics cards (RTX 2070 and GTX 1080 Ti, respectively).

I confirm:

• [ ] that I haven't found another request for this feature. There's #1803. It's from 2018 and was closed in Feb, 2020.
• [✓] that I have checked whether there are updates for my system available that contain this feature already.

Description

Games that support surround sound should also support it when played through Proton (and, ideally, Wine). Examples:

Black Mesa: https://github.com/ValveSoftware/Proton/issues/4545#issuecomment-1143409340 Haven't tested yet if other Source engine games are affected.

Styx: Shards of Darkness: https://github.com/ValveSoftware/Proton/issues/964#issuecomment-1138436039 This game freezes during launch when using HDMI for audio. It works perfectly when using another device (e.g. headphones), but well, stereo only in that case.

Layers of Fear 2: https://github.com/lutris/lutris/issues/4426 I don't have this game on Steam (but GOG), but the behaviour when running it through Lutris is the exact same as Styx: Shards of Darkness, so I presume it's the same bug.

Mirror's Edge: https://github.com/ValveSoftware/Proton/issues/1803#issuecomment-1135784181 No surround sound out of the box, although there is a complex workaround discovered by @Loacoon1 and described in detail in my comment linked above.

Serious Sam 2: https://github.com/ValveSoftware/Proton/issues/1803#issuecomment-1143385877 The Mirror's Edge workaround plus a .ini tweak can be used to get this to work.

Prince of Persia 2008 Works out of the box on Windows but requires IndirectSound on Linux. Doesn't work with DSOAL.

Dishonored: https://github.com/ValveSoftware/Proton/issues/51#issuecomment-1213104231 @ColdPie1 offered a few thoughts on this one, but so far, I haven't been able to deduce a workaround.

Fallout: New Vegas: https://github.com/ValveSoftware/Proton/issues/356#issuecomment-1143392956 No dice, even after trying DSOAL, Mirror's Edge workaround and the like. I read some reports indicating other users are having more success.

Doom Eternal: https://github.com/ValveSoftware/Proton/issues/3773#issuecomment-1213417973 This is probably a false positive. After installing Doom Eternal on my PC, it only played in stereo, but I think I was using Proton-GE. Once I switched to Proton Experimental, it started working. So something might be off here, but probably isn't?

All the above games require no tinkering for surround to work in Windows. Others, like F.E.A.R., Duke Nukem: Manhattan Project or Prey (2006), do require tinkering on either OS, so I don't think they should be listed here (though of course it would be awesome if they worked out of the box through Proton).

Justification [optional]

I understand it's not an important issue. After all, the games I am reporting above are running perfectly well in stereo, and let's be real, who cares about surround sound? I get it, few do.

Still, I'm wary before installing any new game at this point. My failure rates have been pretty high. I'd be happy just knowing more devs are aware that surround sound remains broken in many cases and will eventually look at it. After all, it is a feature of these games that we can enjoy on Windows, but not on Linux. How awesome would it be to just ditch Windows in a couple years?

And yeah, ideally, I would very much hope this could also be added in Wine.

Risks [optional]

Can't think of any.

References [optional]

https://github.com/ValveSoftware/Proton/issues/356#issuecomment-1143392956 https://github.com/ValveSoftware/Proton/issues/4545#issuecomment-1143409340 https://github.com/ValveSoftware/Proton/issues/964#issuecomment-1138436039 https://github.com/lutris/lutris/issues/4426 https://github.com/ValveSoftware/Proton/issues/1803#issuecomment-1135784181 https://github.com/ValveSoftware/Proton/issues/1803#issuecomment-1143385877 https://github.com/ValveSoftware/Proton/issues/51#issuecomment-1213104231 https://github.com/ValveSoftware/Proton/issues/3773#issuecomment-1213417973

[kopr12]

Nice effort and initiative. Currently I'm having this problem with Elite Dangerous, game fully supports it, but all I'm getting from the Proton is stereo. Now, I'm not using the latest Proton (v7 and experimental) because the game doesn't work with it, but earlier Proton versions are stereo.

[Xavron]

Hold onto your seats! lol. What I'm about to say will make certain people angry for even suggesting it who want perfection and true surround. It is merely a suggestion that is good enough for x amount of people.

You can achieve full speaker surround with all channel stereo or my Onkyo also does up conversion to 7.1 should you have bought a receiver that can do that. Its not true surround but it bridges the gap nicely where things fail to support surround.

At least for me the up conversion works fine enough. I'd rather have that than stereo and all the other speakers go to waste. One can be picky and stringent that it has to be true multi channel or nothing. That's up to each one to decide.

I don't know the state of surround with Proton though. Sound is fairly complex. If they have to implement something then depending it can take a while like WMF did or it might be easier. I'm sure one of them working on it have an idea about that.

Little addition: I see multi-channel was added with alsa waay back. https://www.winehq.org/announce/1.5.11 I guess changes since then or who knows... maybe they just didn't work on it and it stopped working. Many don't use alsa directly today I think anymore.

[mrmatteastwood]

Quick update re. Prince of Persia 2008: I was wrong, it doesn't work with IndirectSound. There is sound coming from the center speaker (mainly dialogue), but many other sound effects don't play at all and there is next to nothing in the other speakers either. In the intro, for instance, when the Prince falls down the ledge and a bunch of rocks fall on his head, there is no sound. Right aftewards, when Elika shoves him up against the wall, no sound either. A minute after that, during the first sword fight, there's no sword clanging sound.

@Xavron Yeah, I agree upmixing is better than nothing. Personally, I prefer playing the affected games in true surround using the Windows partition on my laptop, though.

[Xavron]

Yeah, and hence what I stated in the front lol. If you want true then there's a lot a play. Wine added multi-channel to alsa long ago. Probably it broke and is incompatible with various changes.

I'm not sure if anyone here even looked if it can support eg alsa plugin with pipewire or if Wine is just broken.

[mrmatteastwood]

Well, to be fair, many games do work in surround sound using Proton, and it seems more recent ones work better than older ones, too. For instance: Deathloop, Alien: Isolation, Doom Eternal, Divinity: Original Sin DC, BioShock Remastered, Dying Light, even F.E.A.R. 2. They all play in surround sound, so Proton definitely does support surround sound. And I know Wine does, too. It's just still pretty spotty and it looks like it's not one big bug, but many smaller ones, often probably down to a per-game basis.

Although, I'm sure the bug that affects Styx and LoF 2, if that was fixed, probably a whole bunch of other games would start working, too.

[Xavron]

Ah, I see. It could definitely be stuff in the Wine code that hasn't been implemented for specific situations. Looking at the Wine source briefly, I see FAudio does indeed include support https://github.com/wine-mirror/wine/blob/1d178982ae5a73b18f367026c8689b56789c39fd/libs/faudio/include/FAudio.h#L458

So it might be games that can't make use of that. I won't be looking into it any further. But, I'm sure the Proton devs are aware of stuff already.

[kopr12]

Luckily I solved 5.1 in Elite Dangerous, there's a config file where you select which Audio API is used, there are two, WASAPI and XAudio2, so with XAudio2 (which is default) I was getting only stereo, when I changed that to WASAPI I get 5.1 :)

[Hiradur]

In case it helps, regarding Prince of Persia 2008: I ran the GOG version using Lutris Wine builds and also only had stereo output initially. However, when I upgraded to a Lutris Wine build >= 6.13 (it's been a while so I'm not exactly sure about the version, but it should be in this area), surround sound suddenly worked fine. World in Conflict exhibited the same problem and upgrading to a Lutris Wine build >= 6.13 fixed surround sound for it as well.

[mrmatteastwood]

In case it helps, regarding Prince of Persia 2008: I ran the GOG version using Lutris Wine builds and also only had stereo output initially. However, when I upgraded to a Lutris Wine build >= 6.13 (it's been a while so I'm not exactly sure about the version, but it should be in this area), surround sound suddenly worked fine. World in Conflict exhibited the same problem and upgrading to a Lutris Wine build >= 6.13 fixed surround sound for it as well.

Interesting! There's no way to run Steam games through "regular" Wine like you describe, right? I tried through Steam using the latest Proton and GE-Protons, also older versions, with and with out DSOAL and IndirectSound. I also tried launching the games from Lutris using the Steam runner and the options to disable Lutris runtime and use system libraries. I either got stereo or corrupted surround with a lot of SFX not playing at all.

It's good to know that the GOG version works when using plain Wine.

[Hiradur]

It's good to know that the GOG version works when using plain Wine.

I think you might have misunderstood me. I wasn't running the GOG version in plain Wine but rather in the Wine build provided by Lutris, which carries a lot of patches compared to a plain Wine build.

There's no way to run Steam games through "regular" Wine like you describe, right?

It might be possible to run Steam games in arbitrary Wine prefixes by installing the Windows Steam client in the target prefix and install the Windows game from there. Didn't test this, though.

[mrmatteastwood]

Replying to https://github.com/ValveSoftware/Proton/issues/6138#issuecomment-1253739193

Ah, thanks for clarifying. Yeah, it sounds like if I have the game on Steam, not GOG or in some other form, it'll be tricky to run it using Lutris' Wine build. Too bad! Well, gonna wishlist it on GOG, then. Hopefully, Proton catches up some day and fixes surround support for this game and others.

[Bitwolfies]

There's also a chance that if this native Pipewire driver for wine gets off the ground, it could support 7.1/atmos https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1938#note_1499194

[mrmatteastwood]

That would be awesome. I'm guessing Pipewire will be in Linux Mint 21.1 when that comes in December.

This may not be the right place to ask, but for those who have experience using Pipewire: is it the same as Pulse, where we can just plug an HDMI cable into our graphics card and AVR and bada bing, surround sound works? With selectable profiles for 5.1, 7.1 etc in the OS's sound settings? Is there an app that does what pavucontrol does?

[Bitwolfies]

That would be awesome. I'm guessing Pipewire will be in Linux Mint 21.1 when that comes in December.

This may not be the right place to ask, but for those who have experience using Pipewire: is it the same as Pulse, where we can just plug an HDMI cable into our graphics card and AVR and bada bing, surround sound works? With selectable profiles for 5.1, 7.1 etc in the OS's sound settings? Is there an app that does what pavucontrol does?

So long as you install the proper components (Pw-alsa, pw-jack, pw-pulse) then all the audio controls will mix and become one, all your old pulse controls such as pavucontrol will work exactly as they did before, and can now control jack apps.

[mrmatteastwood]

Good to hear. So those are simply additional packages one needs to install, then?

[Bitwolfies]

Good to hear. So those are simply additional packages one needs to install, then?

Yep, the main PW package, and all its other audio servers, just note it will outright replace them, but all Pulse apps and commands should work, such as it auto importing your configs or sinks.

[mrmatteastwood]

For anybody interested, Dishonored and Styx: Shards of Darkness work in 5.1 surround sound with the latest version of Proton Experimental on Linux Mint 21.1. Brilliant!

Will check the other titles I had problems with in due time.

[dsseng]

One of the related technologies is spatial sound rendering. I've been working on one for PipeWire recently. It can easily render 7.1 to headphones (we should be able to do that automatically some day), as well as work with dynamically positioned objects (custom demos are working). However, I've got no real source of spatial sound (not just 7.1, but dynamic sources) to develop a viable API for exposing spatial objects to clients.

I have opened an RFC regarding this: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/3035 and would like to cooperate on creating more immersive audio experiences in games.

Proton/Wine do already provide some implementation for ISpatialAudioClient, but not really rendering it spatially. Proper support for this should enhance user experience and game quality, so comments from Proton developers are welcomed!

[Bitwolfies]

Replying to https://github.com/ValveSoftware/Proton/issues/6138#issuecomment-1435493791

By sources, do you mean games and programs that actually use spacial sound? If so Returnal might be a good start with its Aura 3D support. I don't think it's atmos can apply to headphones however, needs a proper speaker set up.

Had no idea it was you working on the native PW driver, hope to see that soon!

[dsseng]

if an Atmos game uses ISpatialAudioClient, Proton can try to get object props and at least dump them. Then something needs to be done at the driver level, likely requiring WIP Wine-PipeWire driver.

Cyberpunk 2077 has been known to require ISpatialAudioClient, as per initial commit message.

Had no idea it was you working on the native PW driver, hope to see that soon!

Not me, I'm not into Wine at all. Not sure whether the developer is on GH.

[Bitwolfies]

Replying to https://github.com/ValveSoftware/Proton/issues/6138#issuecomment-1435498469

Ah, sorry, I didn't follow the links right.

As for support Aura 3D seems to work fine already in my case, with objects sounding like they're from above or below. I guess its just atmos amongst other formats that require ISpatialAudioClient support.

[dsseng]

As for support Aura 3D seems to work fine already in my case, with objects sounding like they're from above or below. I guess its just atmos amongst other formats that require ISpatialAudioClient support.

Does it just send stereo to headphones and do all the processing inside the game engine? We thought many games did so, but some relied on platform APIs to make 3D audio

[Bitwolfies]

As for support Auro 3D seems to work fine already in my case, with objects sounding like they're from above or below. I guess its just atmos amongst other formats that require ISpatialAudioClient support.

Does it just send stereo to headphones and do all the processing inside the game engine? We thought many games did so, but some relied on platform APIs to make 3D audio

I'm gonna assume internal processing for Auro as its it seems to have two implemenations, one where its just a basic 7.1 signal and one where its a library you can add to games to do spacial audio on headphones with no driver required.

In this games case it chose the embedded 3D simulation , much like how you can run atmos for headphones (no idea if atmos for headphones can accept real atmos tracks) and sending its Auro mix to a real 7.1 setup presumably wont work. Game also never requested any downloads or drivers for Aura to work so its gotta be internally processed.

Games atmos mix is however real and requires a dac and all that fun stuff.

[dsseng]

Atmos mix and Atmos for headphones are ambiguous terms. Typically games are going to use ISpatialAudioClient, not an Atmos encoded file like movies do. So Wine can just see coordinates, transform those and feed into PipeWire spatializer.

Atmos for headphones and Windows Sonic do roughly the same as my PW spatializer: apply HRTFs to make sound appear from a coordinate point

[Bitwolfies]

Atmos mix and Atmos for headphones are ambiguous terms. Typically games are going to use ISpatialAudioClient, not an Atmos encoded file like movies do. So Wine can just see coordinates, transform those and feed into PipeWire spatializer.

Atmos for headphones and Windows Sonic do roughly the same as my PW spatializer: apply HRTFs to make sound appear from a coordinate point

Right, well Auro3d in this case is going with the atmos for headphones approach, just with internal processing, even has a DLL for it. (But it does have its own real codec for 7.1 systems, it's just not used here) But as said, its Atmos Mix is very real and relies on ISpatialAudioClient, sorry about the confusion.

[dsseng]

Alright. Auro3D is then unrelated to the question: it's a module for the game engine that detects sound hardware and performs the best possible rendering on its own, just sending classic 2.0 or 7.1 to the OS audio stack. Nothing to do to support that from Proton and PipeWire. If something is broken and 7.1 doesn't work, that's a bug, not missing feature.

Atmos mix based on spatial client is essentially what I am asking developers about: Proton could probably set those properties to PipeWire (method is to be determined) and PipeWire can render arbitrarily positioned sources to headsets

[dsseng]

The main factor needed for ISpatialAudioClient is collaborative work, I want to hear opinion of developers involved in Wine audio stack to shape the upcoming API

[Brisse89]

Borderlands 2 (49520) and Borderlands The Pre-Sequel (261640) has had broken surround for as long as I can remember, at least on my system. Basically, audio from behind is completely muted but audio from the front plays fine. Finally solved it with protontricks <appid> sound=alsa which gives me working surround audio in these games.

Using Debian testing, with PipeWire (inc. pipewire-pulse and pipewire-alsa) Proton 7.0-6 7.1 virtual surround using PipeWire, going into stereo headphones. Not that it matters. If this isn't working then surely it's not working on a real surround speaker setup either.