X3DAudio1_7_HRTF support

[emoose]

Haven't seen this mentioned anywhere else, but seems https://github.com/kosumosu/x3daudio1_7_hrtf works with RE4 - compiled the latest commit of that (need to edit the DirectX-SDK-June-2010.props files with path to DX9 SDK), copied the compiled x3daudio1_7.dll to Bin32 folder, and with the *.mhr files from hrtf\KEMAR folder placed at hrtf\ next to BIO4\ folder, it should start up & work fine.

(if mhr files aren't placed at correct folder then DLL will cause an exception and show a non-helpful "runtime asked to exit" error, might be why PCGW currently lists RE4 as incompatible with it)

I'm not that great at telling how well the 3D audio is working there though, but seems to be fine for me, no obvious issues with it when I tried it out at least.

The log file the HRTF dll creates mentions that HRTF is missing datasets for certain sample rates though, 12000Hz, 18000Hz & 28000Hz were mentioned in my quick testing, would probably work even better if there was some way to create datasets for them (or maybe convert the audio to a different sample-rate that HRTF supports?)

E: ah seems it's not too hard to create custom MHR files, that PCGW page has a guide for it in the OpenAL Soft section.

Unfortunately the OpenAL-Soft tool for creating them has a lower-limit of 32000Hz built into it tho, but apparently others have been able to just edit the limit in the code for that tool & built MHR files with lower rates fine... I've asked on OpenAL-Soft's github page about it.

E2: Ah well it was a good thing I asked, seems it's likely HRTF wouldn't be as effective at lower sample rates like those. Ideally the x3daudio hook would resample audio first before applying HRTF to it, but doesn't look like it has any support for that. Unfortunately it hasn't been updated in over 3 years now, so not sure how much chance there'd be of adding that in :(

Second best option would be resampling the audio files themselves, assuming nothing is hardcoded in the EXE it should be possible I think, not sure if there's much info about modding RE4's audio files though, will try looking into it some more.

[nipkownix]

Hmm, seems like a good time to talk about some sound issues albert told me about:

• Do you think it's possible to restore the reverb sound effect of certain areas?
• And the positional sound of certain enemies' sounds? (I think steps always sound as they are close to you, or at the least it sounds inconsistently)

I honestly never noticed those issues before. Have you? Maybe simply using X3DAudio HRTF changes something here?

[emoose]

From a quick look it seems there's a SndSetReverb func called by SndRoomStartInit, which then calls into Snd_efx_req

On GC the Snd_efx_req func can do a few different things, either run efx_req_set_new, efx_req_set_update, or efx_req_stop But on PC it seems only the efx_req_set_new call is left - with all the same checks for it beforehand etc, implying PC is maybe only ever able to create an efx effect, but not update or stop it.

The PC efx_req_set_new code looks pretty simple, maybe could just hack it so that also gets ran when it wants to update too, but not sure if the params that it changes actually get used for anything or not.

Seems XAudio2 does have a reverb effect that can be added, but no idea if game tries making use of it at all yet (not too sure about how to reverse COM stuff like that, pretty sure IDA has some support for it, need to look into it more)

(E: seems the XAudio2 effect isn't being used at least, game does have the CLSID for it built into the EXE but doesn't get used by anything...)

About enemy sounds, HRTF did seem to help with that a bit I think, haven't tried it out that much though.

E: ah darn, seems the HRTF hook doesn't play well with movies, causes another "runtime requested to exit" error whenever one tries to play, I'll try seeing if I can track down the cause tomorrow.

Also got another runtime error during gameplay, not sure what caused that though.

In happier news I managed to write a tool that can resample the XWB audio files for us, seems to work fine across all the games audio (in the Snd folder at least), and gets rid of the "missing datasets" log warnings, so should hopefully help HRTF get mixed into even more sounds.

[emoose]

HRTF plugin has been updated to work better with RE4 😃 https://github.com/kosumosu/x3daudio1_7_hrtf/releases/tag/v2.4

It includes MHR files for the lower-sample-rate audio too, but from what OpenAL-Soft dev told me the HRTF effect would be improved if the audio is resampled to something higher first, x3daudio1_7_hrtf doesn't support on-the-fly resampling unfortunately, but the XWB format the game uses to store audio isn't too complex, so I've made a resampling tool here: https://github.com/emoose/re4-research/blob/master/re4resample.cpp

(build at https://github.com/emoose/re4-research/releases/tag/0.1)

To use that you'd just run it like re4resample.exe -r 48000 bio4evt.xwb, and it'll go through the bio4evt.xwb file and resample any audio entries that are lower than 48000, make sure you have ffmpeg.exe somewhere in your path (or next to the re4resample.exe file) since that's used to actually do the resampling for us.

(alternatively you can copy re4resample.exe, auto-resample.bat & ffmpeg.exe into your Snd folder, then run auto-resample.bat, and it should process all the xwb files for you)

Some xwb files contain loop-points which I'm not fully sure how to handle, there's a -l switch you can supply to re4resample for it to try updating the loop points, but I'm not sure how accurate that is, if you don't specify -l then it'll just skip over any entries that use loops.

[nipkownix]

Albert has been doing some tests with this, and this is what he had to say:

I've been messing with all this stuff and there is some info I'd like to share with you and emoose, just in case:

Some low frequency sounds will sound worse ingame if they are converted to 48000 hz. It seems the game has an internal method to resample low frequency sounds and it makes them sound way better than if you simply hear the raw wav file. I attach a sample (sound and visual evidences) But if the sound is already 48000 hz, the game won't do anything and it will sound as is: muffled.

This is applicable to all em##.xwb sounds (the enemy sounds). In fact, it seems this method is better than in later ports of the game. In the PS4 port, for example, enemies sound worse... On the other hand, the game doesn't apply this resample filter to stage sounds (r###.xwb). I have no idea why the game doesn't apply the resample filter to all the sounds...

Not sure about other contexts. The way of knowing what the game does is to record ingame sound and see the resulting wav in spectral frequency view in adobe audition and compare it with the original sound like I did in the attached example:

I haven't noticed the issues he mentioned, but maybe XAudio is doing some magic of its own in the low frequency sounds?

He also mentioned some crashes with the latest HTRF mod:

Also, the x3daudio1_7.dll file (xaudio_hrtf_2.4_Win32) makes my game crash when certain sounds play (a breaking barrel, for example) and the resampled sounds are worse.

Not sure what could be going on here.

[zoluxsys]

Does someone has the binaries or source code for the X3DAudio.dll? The repo is not available anymore... https://github.com/kosumosu/x3daudio1_7_hrtf/releases/tag/v2.4

[emoose]

Does someone has the binaries or source code for the X3DAudio.dll? The repo is not available anymore... https://github.com/kosumosu/x3daudio1_7_hrtf/releases/tag/v2.4

Huh that's weird, wonder what happened there.

Found a couple things I still had saved, here's the original releases from kosumosu: xaudio_hrtf_2.4_Win32.7z.zip xaudio_hrtf_2.4_x64.7z.zip (rename .7z.zip to .7z, github doesn't support 7z filename...)

and here's a custom build with a minor fix included, might help allow it to work with more games (fix is detailed below): https://github.com/nipkownix/re4_tweaks/issues/24#issuecomment-1104041991

Also have some code for it dated 2021/11/29, not sure if it matches with 2.4 or not though (2.4 dll is dated 2017 so I guess it does) x3daudio1_7_hrtf-master.zip

[zoluxsys]

Thank you!

[emoose]

Oh just realised, if that DLL is from 2017 then it's missing a fix that RE4 needed to stop crashing with certain sounds, unfortunately can't remember all the details for that now though, I did write a issue page for the dev that detailed it, and they did eventually fix it in the source code too, but I guess they forgot to update the DLL in that 2.4 release :/

E: fortunately one of those github-clone-sites seem to have cached the issue report I made:

(the CreateMasteringVoice link mentioned there was to x3daudio1_7/xaudio2-hook/graph/AudioGraphMapper.cpp line 141)

@nipkownix maybe this could be the cause of the crash albert mentioned

the x3daudio1_7.dll file (xaudio_hrtf_2.4_Win32) makes my game crash when certain sounds play (a breaking barrel, for example) and the resampled sounds are worse.

Doesn't mention movies there but barrels breaking, if the 2.4 release is using a 2017 build of x3daudio I guess it could be missing a lot of other fixes maybe. I'll try seeing if I can build the source I saved with that fix included.

[emoose]

Here's a build with the fix above included, this is mainly for RE4 but might also help with other games that crash when using X3DAudio-hrtf, for RE4 just extract the zip to RE4 root folder (except Bin64 dir), for other games check the README.txt

x3daudio_hrtf_2.4a_x86-x64.zip

(this might only work with Win10+, as I can't install Win7 SDK to build with...)

[emoose]

Just saw that there's a github repo with the x3daudio 2.4 changes up at https://github.com/ThreeDeeJay/x3daudio1_7_hrtf/releases/tag/v2.4.1.0.28, looks like that has the git history up to just before 2.4 release as well, so probably better to use that if anyone wants to mess with the x3daudio source.

[ThreeDeeJay]

Just saw that there's a github repo with the x3daudio 2.4 changes up at https://github.com/ThreeDeeJay/x3daudio1_7_hrtf/releases/tag/v2.4.1.0.28, looks like that has the git history up to just before 2.4 release as well, so probably better to use that if anyone wants to mess with the x3daudio source.

@emoose Hi, I'm the one who made that fork so I think I should point out:

• It's a fork that merged the last official (2.4) update from the now-made-private original repository by kosumosu and another fork that implemented a handful of QoL changes, fixes and a working appveyor workflow you can set up so every commit generates binaries automatically on the cloud
• The most up-to-date, unmodified fork of the original repo is https://github.com/clayne/x3daudio1_7_hrtf/commits/master, for future reference