[XAudio2] FAudio support

[flibitijibibo]

So now that this is out in the wild...

FAudio is a new, accuracy-focused reimplementation of XAudio2 and its relative libraries (such as X3DAudio, XACT, and XAPO). Part of this work includes a COM wrapper that mimics the Windows XAudio2 DLLs, allowing Windows games to use FAudio for audio support instead. This is known to work on both Windows and Wine for 32- and 64-bit, and both @JohanSmet and @aeikum have been putting in work to make it more compatible with the whole range of XAudio2 versions.

At the moment Wine is reimplementing XAudio2 with OpenAL Soft, and the other libraries (including X3DAudio) are currently stubbed. In addition to the feature gaps, there are also various accuracy issues that come from wrapping a low-level audio API around a high-level API (as we found ourselves while making FNA, hence the creation of FAudio).

The idea is that FAudio would both fill in large gaps left by the current Wine implementation as well as make the existing implementation much more accurate, and in a perfect world, remove the need for the DX redist for audio support.

Andrew has been leading the charge with Wine compatibility lately so I'll let him drive this thread, but the main things that Wine cares about:

• [x] WMA support via FFmpeg
• [ ] XAPOFX support
• [ ] Full X3DAudio support
• [ ] Possible (read: real) support for CommitChanges
• [ ] Possible support for unaligned buffer data (see: Super Meat Boy ADPCM)
• [x] XACT DLLs (two example games: Resident Evil 4, P.B. Winterbottom)
• More...?

[flibitijibibo]

Spent some time on this today, the COM wrapper should work out of the box again (some recent 2.8+ compat and SIMD optimizations screwed it up) if anyone wants to play with this right now.

[mirh]

At the moment Wine is reimplementing XAudio2 with OpenAL Soft, and the other libraries (including X3DAudio) are currently stubbed.

Uh? For real, no more MMDevAPI? And.. are we sure this time the third or fourth rewrite of the audio stack will finally be the right one?

and in a perfect world, remove the need for the DX redist for audio support.

I mean, isn't that what windows itself requires?

[flibitijibibo]

FAudio is a plain-as-peaches C implementation of XAudio2 and is already shipping in other work, so I'm pretty sure this won't end up like other scenarios where implementations were done with high-level wrappers or were unfinished and abandoned. Windows does require the DX redist but we don't have to be like that!

In other news, I just threw together the XACT COM wrapper source, but not the project files themselves... I think it'll work but the COM stuff isn't something I'm terribly familiar with myself:

https://github.com/FNA-XNA/FAudio/issues/27

[aeikum]

Just so no one goes too far down this road, I already have the Wine integration work for xaudio2, xapofx, and x3daudio done in my tree. It's currently at the point where I'm testing games that worked in Wine's "old" xaudio2 and ensuring they work in the new FAudio-based xaudio2. I just got busy with other stuff ;)

[mirh]

So.. How's the final picture going to look like?

Windows does require the DX redist but we don't have to be like that!

Mhh.. Newer windows maybe not either - but I'm not sure how much wine would be happy to differ otherwise.

[aeikum]

Same as it is now, we're just going to use FAudio to do the audio mixing instead of OpenAL, since it's a much better fit. All the audio still goes through the normal Wine audio paths.

[mirh]

Oh, "reimplementing" was with respect to windows, not a previous wine architecture.

Are you also going to implement directsound on top of it then? Vista+ does at least. How's FAudio going to play out with XP setting then? Are you going to add it a (faithful) directsound backend?

[flibitijibibo]

Just finished making the XACT DLLs! XACT 3.0 to 3.7 should work with FAudio now.

EDIT: Demo vid, because why not https://www.youtube.com/watch?v=_gFpDfN60S4

[tgurr]

@flibitijibibo would it be possible to have the dlls in a precompiled form ready to download somewhere so the guys from winetricks could add a verb for it to ease testing?

[flibitijibibo]

At the moment FAudio is moving really fast so precompiled wouldn't be of much use - I may do this for stable releases though (expected to be every month on the 1st, alongside FNA releases). This may be a good time to test out Johan's README:

https://github.com/FNA-XNA/FAudio/tree/master/cpp

[aeikum]

FWIW I put my WIP patches at https://github.com/aeikum/FAudio/commits/wip-totally-broken . It is in no way usable in its current state, but has some needs-more-research fixes to get more games to work.

[aeikum]

I also put the matching wine.git commit here, again, totally broken. https://github.com/aeikum/wine/commits/wip-faudio

[flibitijibibo]

On the subject of broken things, the XACT wrapper is now at the point where Resident Evil 4 should run (though it's probably very fragile since it depends on a slightly unimplemented function). If you self-build, set this to 1:

https://github.com/FNA-XNA/FAudio/commit/38471d2267f8f63598d367c984f0123a7fe54177#diff-396a3a789a898fb29cce92cb6fc31014R2328

[flibitijibibo]

By far the most requested feature since Proton has gone out is WMA support - here's the issue where we're now tracking this:

https://github.com/FNA-XNA/FAudio/issues/32

There's some existing code but none of it is meant to work (it doesn't even compile yet), so if you're familiar with FFmpeg and want to fix this problem, that's the place to start.

[flibitijibibo]

Work on this has been sparse since FAudio now works with FNA and I'm still in the process of seeing if I can work on this in a more official capacity, but in the meantime I've been given a couple games and I made some more samples.

Sonic Mania: https://www.youtube.com/watch?v=FV_GPY8O9ww

A Hat in Time (64-bit!): https://www.youtube.com/watch?v=I9NEkqfhWDw

[flibitijibibo]

Had a bit of free time today, so I cleaned up the existing work for the COM wrapper and now it's MUCH easier to build and run yourself. All you need is MinGW-w64, MinGW-SDL2 (2.0.8 or greater), and Git, all of which are most likely in your repositories.

To build:

git clone git://github.com/FNA-XNA/FAudio.git
cd FAudio
source cpp/cross_compile_64 # Can be 32 if you want
make
cd cpp
make

Then you can use cpp/scripts/wine_setup_native to install these to a specific prefix. Be sure to read the documentation and the script first, however:

https://github.com/FNA-XNA/FAudio/blob/master/cpp/README.md

As of now my recommendation is to install the XAudio2 DLL, but NOT X3DAudio. So you would be removing these 5 lines, for example:

https://github.com/FNA-XNA/FAudio/blob/master/cpp/scripts/wine_setup_native#L62

XACTEngine may or may not work, but honestly so few games use this that I'd be surprised if you ran into it at all.

[benoit-pierre]

There's a typo in the cpp/README.md: scrips/cross_compile_32 -> scripts/cross_compile_32. It would also be better if those 2 scripts used return 1 instead of exit 1 when the mingw32 / SDL environment is incorrectly set, since they are supposed to be sourced.

[flibitijibibo]

If you can, do a pass through the files and for anything you find that's off send a PR, I can merge it right away if it's all docs plus the return code.

[dhewg]

Tested on Witcher 3, which fixes audio cracks followed by volume drops I got with wine internal xaudio2_7, nice!

[pc-bil]

Tested Skyrim Special Edition but without success. The following errors were logged: INFO: Invalid destination channels WARN: \n\nAssertion failure at FAudio_PlatformInit (src/FAudio_platform_sdl2.c:188), triggered 1 time:\r\n '0 && \"Failed to open audio device! WARN: \n\nAssertion failure at FAudio_CreateSourceVoice (src/FAudio.c:260), triggered 1 time:\r\n '0 && \"Unsupported format tag! WARN: \n\nAssertion failure at FAudioSourceVoice_SubmitSourceBuffer (src/FAudio.c:1274), triggered 1 time:\r\n 'pBufferWMA == ((void *)0)

So, it's not looking too promising.

[flibitijibibo]

Output filters and tail playback have both been implemented, covering some of the obvious (but not critical) features used by a handful of games (PAC-MAN CEDX+, Rayman Origins). WMA support still isn't in so Skyrim/Fallout players get to wait a little bit longer until that's done.

The OP has been updated with some more tasks that Wine is interested in.

[romulasry]

Any guess on the ETA before submitting to Wine/Proton? Thanks in advance. Very promising project IMHO.

[ryao]

There is an update on Twitter:

https://mobile.twitter.com/flibitijibibo/status/1051967505160388614

[ibrokemypie]

Anyone know what is required for audio to work with hellblade? It doesn't currently launch when using audio dlls

[flibitijibibo]

While discussion continues regarding integration in Wine upstream, I've been playing with our COM wrapper a bit more and found that some new extensions we made for Wine also helped make built-in support possible for our own wrapper. As of the latest commit, FAudio can now generate .dll.so binaries for all of XAudio2, XACT, X3DAudio, and XAPOFX. Note that F3DAudio is missing some features and FAPOFX is one giant stub until this issue is finished.

This simplifies the build process slightly because instead of needing a whole MinGW setup, now you just need SDL2-devel, wine-devel, and ffmpeg-devel. That's the next big change: We finally got WMA support in! It's highly experimental but if you build FAudio with FAUDIO_FFMPEG=1 make you should get an FAudio build that uses and links to your system's FFmpeg. I already know of at least one game that does not work, and FFmpeg's WMA support seems to have some gaps as well, but now we can gather a little more data on the ones that haven't been tested yet.

[flibitijibibo]

Here's a testing-only patch that rewrites the XAudio2 implementation to use FAudio, for anyone building Proton from source:

https://github.com/ValveSoftware/wine/pull/32

For those using upstream wine, here's the patch you care about:

https://github.com/flibitijibibo/wine/pull/1

[flibitijibibo]

Proton 3.16-5 ships with FAudio support! It should cover most cases but the one big case is WMA support. This is a licensing restriction that's out of our control right now, but thankfully it's still in yours:

https://github.com/FNA-XNA/FAudio/wiki/FAudio-for-Proton

Build against your own system's FFmpeg and make sure Steam uses the system libraries instead of the Steam runtime's and it should work with no trouble.

[mozo78]

This is wonderful news! Is the FAudio support is enabled for default or it's needed some user actions?

[flibitijibibo]

It’s enabled by default, but WMA support is disabled.

[juppso]

First up, this is great news and great work, I was just wondering - are there any plans/potential methods to work around the licensing issues of FFmpeg? Or is this just something we have to live with?

[flibitijibibo]

I found out today that the Steam runtime has FFmpeg, so I’m going to see if we can’t just link to that and make it so WMA support is as simple as dropping your own WMA-enabled FFmpeg into the runtime, instead of having to do a whole rebuild. But yeah, the legal issue is something we’re stuck with.

[juppso]

From that I assume the steam runtime FFmpeg doesn't have wma support, is that due to the same licensing issues? How do distro repo's get around that?

[flibitijibibo]

You’d have to ask each distribution about it. I know Fedora doesn’t provide FFmpeg at all due to issues like this (though you can get it from third parties), and other distributions with strict software policies are probably like this too.

[BeerZ0rg]

yup, Fedora doesn't provide FFmpeg due to legal issues, although negativo17 multimedia repo provides one with every codec enabled i believe. It sucks that WMA can't be enabled, i really hoped for Skyrim: SE (which now crashes) to be working out of the box.

[mozo78]

It’s enabled by default, but WMA support is disabled.

Thank you :)

[arendtio]

I just tested it (for about 3 minutes) and it seems to work :+1: (Skyrim SE + FAudio + FFMPEG with WMA support). The one thing that took me a few minutes to figure out, is that the documented copy command isn't foolproof ;-)

I had to add \ Beta:

cp libFAudio.so ~/.local/share/Steam/steamapps/common/Proton\ 3.16\ Beta/dist/lib64/

Not very surprising given the fact that 3.16-5 is still in Beta ;-)

[dragonfly-lover]

Strider uses xaudio2, and sadly audio doesn't work yet.

[ghost]

Proton 3.16-5 ships with FAudio support! It should cover most cases but the one big case is WMA support. This is a licensing restriction that's out of our control right now, but thankfully it's still in yours:

Thanks! you guys are doing gods work, latest release makes skyrim playable

[mirh]

Isn't this expired already? And even if not, wouldn't it fall under the 60k patents microsoft opened 2 months ago for OIN?

[juppso]

Assuming it is that patent which is in question for this case, it would appear that while not expired it should be covered under the OIN?

[BeerZ0rg]

I have FFmpeg 4.1.0 with all codecs enabled (including WMA), i've built FAudio and replaced it in Proton 3.16 Beta and now Skyrim: SE runs, but there is something wrong with sound. It is buzzing or cracking every few minutes. Also Skyrim crashes on exit (same way it did with xact native dll).

[kakra]

@BeerZ0rg I'm no longer having the crackling sound issue in my build. Could you try https://github.com/kakra/wine-proton, it is rebased to a newer wine version including several patches and experimental improvements. Maybe we can isolate the required patches... SkyrimSE works fine here. But currently you'd have to compile from source (should be easy) and extract the resulting dist.tar.xz to the proton beta folder.

[BeerZ0rg]

@kakra Thanks, i'll try this when im back home in couple of hours. The ammount of work done here is so impressive im feeling like Christmas already.

[kakra]

@BeerZ0rg Nah, there's a few days left until then. ;-)

I'm planing to push a refined branch this evening (CEST).

[foresto]

Also Skyrim crashes on exit

Skyrim has a crash-on-exit bug caused by one of the DLC. (Dawnguard, I think, but I'm not sure of that.) You might want to try again with DLC disabled.

[kakra]

@foresto It can crash or freeze. Usually, with built-in xaudio2_7, it freezes. It may crash otherwise with native xaudio2_7, I'm not sure, but from Proton it looks like a clean exit so that's okay.

[kakra]

@BeerZ0rg My branch is updated now.

[atm-florian]

As explain in https://github.com/FNA-XNA/FAudio/wiki/FAudio-for-Proton SDL 2.0.9 is needed... Ubuntu 16.04 (or even 18.04) => SDL 2.0.4 (2.0.8)... Can't compile... Arfff, in one or two year Fallout 4 or Skyrim SE should be played out of the box with Proton. But for now stuck with Xaudio.

[kakra]

It compiles fine here on Gentoo with SDL 2.0.8, tho it's throwing a warning during compile to upgrade ASAP. In my Ubuntu (cosmic) container, it also compiles fine, cosmic seems to have SDL 2.0.9.

[mati865]

@kakra Debian sid and Ubuntu cosmic/disco still are at 2.0.8 version. Once Debian upgrades to 2.0.9 it'll to be pulled by next Ubuntu release before freeze (it's too late for anything <19.04 aka Disco).