Proximity/NF Effect

[ABoredBunny]

This is a feature request for a near field effect to be added.

I believe it would add a lot of depth towards binaural playback (And Speakers, depending on how it is implemented).

It could be toggled by a developer, but giving end user the option to force it on in alsoft config.

This could be implemented in many ways, but even just an EQ effect would be fine.

[kcat]

There already is one technically, but other than with HRTF, it depends on the user setting up a speaker configuration with the proper speaker distances set, as well as the app using appropriate distances in the game.

I have been thinking it needs a bit of a rework though, since most apps don't use appropriate distances or set a proper distance scale (and not all of the needed info can be provided by the API currently), and most users won't know or necessarily want to set up a modified speaker configuration to set the speaker distances.

[ABoredBunny]

I meant one that worked on headphones.

I would be fine setting distance scale per application, using something like a .json file, for backwards compatibility. While newer programs can set it themselves.

Also, what data is missing from the API? Don't you just need relative "object" distance (when distance scale is available)?

[kcat]

Also, what data is missing from the API? Don't you just need relative "object" distance (when distance scale is available)?

The perceived scale for the player (how many in-game units relate to a real world meter), which can be different from the in-game meter scale. Basically how many in-game units should sound like it's one meter away to the player in their room, separate from the game world distance. Certain sounds may also want a different base distance for the near-field effect (B-Format buffers can have a near-field effect encoded into it, so properly handling that can make it sound better).

[ABoredBunny]

The first one can be applied using the aforementioned ratio. The other one is more tricky. Something like a sound radius seems like a "simple" solution. Where, for example, a rocket launcher missile would have a smaller radius than a big monster. But an option to force near field on, with an option to set ratio, and a single sound radius for everything(as an example, stuff using dsoal), would be nice.

[ThreeDeeJay]

CMSS-3D's MacroFX could be used as reference in RightMark 3D Sound positional test: https://youtu.be/XJiKbjCuPro?t=19

[kcat]

MacroFX works somewhat differently than OpenAL Soft near-field effects. As I understand it, MacroFX specifies a 0...1 factor on a source, where 1 is normal and 0 is "very close". OpenAL Soft's handling works as a function of pure distance, ranging from infinity (a plane wave) to 1/4th the speaker distance (don't want to simulate it being too close relative to the speakers, or it'll get really loud and bass-y), providing a more thorough modeling for near and far sound sources.

The MacroFX factor alters the sound from being at the speaker distance (1) to some arbitrarily near point (0), generally when within the sound's reference distance, whereas OpenAL Soft tries to create a consistent perception of all distances (with the output properly configured, a sound 1 meter away will be modeled 1 meter away regardless if the speakers are half a meter away or 1.5 meters away, and regardless of the source's reference distance).

Also for HRTF, the near-field effect only works when using one of the ambi1, ambi2, or ambi3 render modes (in the HRTF tab of alsoft-config, or the hrtf-mode config option), rather than full.

[ABoredBunny]

I was wondering, more about, a near field implementation that works on FULL HRTF. With distances up to whisper distance, if need be with some mixing, like making the overall mix quieter.

Something to make it more like a binaural recording.

[ThreeDeeJay]

Full HRTF NFC in OpenAL Soft would be fantastic, though if you're looking for an ASMR-like mixer, you might wanna look into Anaglyph VST DAW plugin or the 3D Tune-in Audio Toolkit app in the meantime.

[ABoredBunny]

I was already aware of Anaglyph (it does have some nice tricks worth investigating, like rendering sound from the ear's perspective to reduce parallax), but I wasn't aware of the 3d tune-in stuff, so thanks for that. I will probably use it for personal projects.

But, just to be clear, I'm interested in it for more immersive sound inside video games. Things like hearing a rocket pass by your ear and hearing how it missed you by a hair's length. Being jump scared by an enemy coming from behind (like a creeper, or an Enderman in Minecraft). Improved distance judging, etc.

[kcat]

OpenAL Soft supports multi-field HRTFs, that is HRTFs that model near and distant responses, and it will select a distance like it adjusts the NFC filters based on distance. There aren't that many HRTFs I know of that measure both near and distant responses (as opposed to all near), but the SCUT KEMAR HRTFs are ones that I know do. The SCUT_KEMAR_radius_all.sofa should work with the makemhr utility to create an mhr for OpenAL Soft, which you can then set as the default HRTF.

As for synthesizing near-field effects with normal (non-multi-field) HRTFs, that will be more complicated since I don't really know the math behind it. With ambisonics (at least at the orders OpenAL Soft deals with), it ends up as a few well-known shelf filters that can be applied to the directional components of the soundfield, but the function is different when applied to discrete HRIRs, and it's not something I've had a chance to look deeply into.

That said, the ambisonic HRTF render modes aren't that bad. Even at first-order/ambi1 it maintains full 3D positional awareness, with second- and third-order/ambi2 and ambi3 being progressively sharper and less diffuse (simulating 8, 14, and 20 speakers, respectively). And it results in consistent handling of the core mixer effects (including angular spread), leaving the HRTF itself to a relatively efficient post-process.

I've actually considered removing the Full HRTF option in favor of the ambi3 mode because the Full mode works so differently and requires special-casing a number of operations, often in poorer and more CPU intensive ways; like gain and panning interpolation (which requires interpolating the whole FIR filter, without the easy-out checks of nothing changing), angular spread (which is rather poorly simulated), and near-field effects (which is completely missing, relying on the HRTF itself having it). As well, things like B-Format sources (including UHJ and Super Stereo) and EFX/EAX effects have to go through the normal mixing routines anyway, so it can't even save on skipping that.

[ABoredBunny]

And I was wondering why the SCUT KEMAR .mhr was that big... Now I know.

Artificial near field isn't necessary, when you have working multi field.

Just a question regarding that, could you add a distance multiplier overwrite? So adding something like "Distance_Multi= 0.5" to alsoft.ini per game, in case the distance isn't right.

Also adding an option to include multiple sofa files in makemhr, would likely make it easier. To use multi field, unless that is already a thing.

A question, how hard would it be to increase the ambisonic order for headphones? 3rd order should be fine for speakers, but headphones benefit from higher orders.

[kcat]

Just a question regarding that, could you add a distance multiplier overwrite?

There's the nfc-scale config option under the game_compat section: https://github.com/kcat/openal-soft/blob/1ad553b5a913e11c712eb95b7ecc7cbc66f4a274/alsoftrc.sample#L599-L611 You need to set it per-game since it depends on the unit scale the game uses.

A question, how hard would it be to increase the ambisonic order for headphones? 3rd order should be fine for speakers, but headphones benefit from higher orders.

4th order might be doable, but 5th order and higher would need much more extensive work since it would exceed 32 channels (which is a limit due to using 32-bit integers for certain bitfields), and I would have to do research for the encoding coefficients and NFC filters in the higher orders. There would also be performance concerns from mixing and processing that many output channels per source with more complex shelf filters.

[ABoredBunny]

That is good, but do you have any idea how I could get makemhr to accept multiple sofa files, or merge multiple sofa files, because most multifield hrtf seperate them by distance?

I would be fine with 4th order, Headphone only if that is easier to implement.

Beyond 4th order I have to listen carefully for differences.

[kcat]

That is good, but do you have any idea how I could get makemhr to accept multiple sofa files, or merge multiple sofa files, because most multifield hrtf seperate them by distance?

Not easily. You'd need to create a .def file that lists each individual response you want in the resulting mhr file, specifying which sofa file to use for each response. Currently you can't simply provide multiple sofa files as input to treat them all as one large sofa file.

[ABoredBunny]

nfc-scale doesn't seem to work for multifield hrtf, is the option only for speaker?

Also, do you have a guide on the file structure of .def? I have no problem writing a file like that as long as it is something like "[0.5]Dist=ARI1424_0.5m.sofa"

[kcat]

nfc-scale doesn't seem to work for multifield hrtf, is the option only for speaker?

No, it's used for HRTF rendering too (both full and ambi modes).

Also, do you have a guide on the file structure of .def?

See MIT_KEMAR_sofa.def. A more thorough description can be seen in MIT_KEMAR.def (although it hasn't been updated to mention how to use sofa files).

[ABoredBunny]

So if I understood the SCUT_KEMAR.def Right.

rate = 44100

points = 512

radius = 0.09

distance = 0.2, 0.25, 0.3

azimuths = 1, 24, 36, 72, 72, 72, 72, 72, 72, 72, 36, 24, 1; 1, 24, 36, 72, 72, 72, 72, 72, 72, 72, 36, 24, 1; 1, 24, 36, 72, 72, 72, 72, 72, 72, 72, 36, 24, 1

[ ] = sofa : "./SCUT_KEMAR_radius_0.2.sofa" [ ] = sofa : "./SCUT_KEMAR_radius_0.25.sofa" [ * ] = sofa : "./SCUT_KEMAR_radius_0.3.sofa"

This is how it would look like, if I tried to make an HRTF using multiple sofa files(using unpacked SCUT_KEMAR)?

Or do I need to do something special so it understands what to do with multiple files?

[ABoredBunny]

Also, I thought nfc-scale didn't work because you just implemented it in the last five days, and I was still on 1.22.2.

Damn, you work fast, thanks for adding that.

[kcat]

This is how it would look like, if I tried to make an HRTF using multiple sofa files(using unpacked SCUT_KEMAR)?

Possibly. Sofa support is a rather late addition for makemhr, added by someone else, and I'm not quite sure how the [ * ] for sofa files will act with multiple sofa files, but a quick look at the code initially appears that would be correct.

[ABoredBunny]

Okay, all the near field stuff is cleared, thank you very much for taking the time to help me. And adding that Near field multiplier.

I do hope in the future, that you add higher order ambisonics, like 4th order. But that is largely outside of the scope of this feature request.

So before I close this I just wanted to say that I very much appreciate your work, And hope you have a very nice day.