Use separate AudioListener nodes for direction and distance attenuation

[Norodix]

Describe the project you are working on

Third-person (TPS) 3d game with lots of positional audio.

Describe the problem or limitation you are having in your project

It is difficult to achieve a good sounding positional audio with third-person games.

There is a discrepancy between whether the audio listener should be placed on the player character or the camera.

If it is placed on the camera it creates weird sound spikes when the camera gets close to an audio source even though the player did not move at all.

If the audio listener is placed on the player character the sounds' direction feels unnatural. Hearing a sound from exactly right from the player is actually a sound source directly in front of the camera (and a little bit to the right).

Describe the feature / enhancement and how it helps to overcome the problem or limitation

In this reddit thread a GW2 developer describes the same exact problem and their solutions.

https://old.reddit.com/r/gamedev/comments/gwv8yw/3rdperson_games_should_positional_audio_be_from/

The relevant part:

In Guild Wars 2, I tried something rather different, and was able to do this because I wrote our own audio engine, built off the low-level FMOD API. Instead of simply designating a single listener position used for all source calculations, we split the listener into two components, a transform for relative angle calculation of sources, and a separate position for falloff distance calculations.

So, we calculate audio falloff distances based on the player position, and calculate the relative angle of sound sources (resulting in 3D panning) based on the camera's transformation (both angle and position). In this way, if a sound source is right next to the player, the angle at which it plays relative to your camera seems logical, and yet the volume falloff is controlled by how close the source is to the player's avatar, not the distant camera.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

A possible solution is separate nodes for AudioDirectionListener and AudioDistanceListener.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, there is no way currently that can separate these two properties. (As far as I know)

Is there a reason why this should be core and not an add-on in the asset library?

I do not think this is possible to do outside of the audio engine.

[yosoyfreeman]

I personally think that splitting the listener in two nodes is going against what they are supposed to do.

This problem do not only exist in this case, but also in any case. Audio falloff is just a curve, but creating an actual audio system for your game is exactly the same as creating any other mechanics. You need to implement sound propagation or sound decreasing based on obstacles yourself, for example. This is done differently depending on the game, as there is no global solution and depends heavily on what you want to prioritize.

In terms of this specific case: I think the Guild wars way is not good as you want sounds to come in camera space, this is the same for movies, you don't want to create dissonances.

BUT if that is what you want, there is no need for using low level stuff and a custom sound system. That is a super complex solution to a simple problem.

You just want to place the listener in your player, but rotate the transform of the audio listener to align with you camera direction.

That way sound falloff is based on player distance, but direction is based on camera direction. Is exactly the same result and much easier and cheaper.

Hope this helps.

[Norodix]

I don't think you assessment is correct.

Imagine the sound source is right next to the player to its right. Then in your solution the sound would be coming from the right. But the intention is that the sound should be coming from the front, because the sound source is right next to the player character which is obviously in front of the camera.

If you have an idea to implement this without modification to the engine I'm all ears but this is not it.

From the same comment:

In games where the player's avatar can extend a long way from the camera, especially when the camera can move around freely, putting the listener on the camera can cause problems with identifying important sounds that are near the player, which is, after all, the most important focal point of the game.

I understand that some people prefer sound in camera space but this method is clearly used in games to improve them.

I think this is a useful feature that is currently missing.

[yosoyfreeman]

Hi, sorry if i was not clear. But what i said would do exactly what you are asking to do.

3D audio listeners are, as their name suggest, 3D. If you just put a listener inside the character then yes, you have variation on the angle based on it's rotation. But if you set it as top level it will not inherit the transform of its parent node. So you can set its correct position on the player (To have falloff based on that) and then replace the listener transform basis with the basis of your camera each frame. This will align the listener rotation with the camera instead of the character.

[Norodix]

In your description the listener is on the player character. The sound source is close, it will play loud. The sound source is on the right. Stereo will play directly from the right.

In my description the distance listener is on the player character. The sound is close, it will play loud. The direction listener is on the camera, the sound source is in front a little to the right (blue arrow). Stereo will mostly play from the front.

I am not worried about the player's rotation. I am worried about its relative position.

[yosoyfreeman]

Oh, i see, i misunderstood what were you trying to do. Sorry for that. Was trying to offer a temporal fix at least. In that case i would not use 3d listeners and emitters but normal ones and manually panning their sound based on camera dot product with the emitter sound and control the volume based on distance with the player. Hope that can help at least until you get a better solution.

[Norodix]

No problem, I'm happy the issue/proposal is clear now!

I do not see a way to affect how the AudioStreamPlayer affects the different audiochannels, so I don't think it is possible to pan it like that. Maybe using an AudioStreamPlayer3D and panning it but disabling its attenuation and setting it from script could recreate this effect perfectly. It seems possible but very hacky.

[yosoyfreeman]

You have built in panning so at least that part you don't have to make it. Good luck!