justinng
commented
9 months ago Was this ever addressed? I'm not using the C API, but I wonder if I'm running into a variant of this issue.
I'm using Steam Audio with Unity/FMOD, and have a working scene where an audio source playing a sustained sound (set up with a Steam Audio Spatializer in FMOD) will seem to pan around correctly if moved around relative to the listener. However, if I switch it to play a short footstep sound, the initial transient will very noticeably play from stereo center, while the tail seems to pan smoothly but quickly to the right position. This happens every time the FMOD event is triggered again (from the StudioEventEmitter.cs component).
However, if in the FMOD Steam Audio Spatializer I turn on "Apply HRTF To Direct", the entire sound plays with proper positioning.
It seems that the SteamAudio uses some sort of "interpolation" for every object involved in a simulation, in a sense that sound sources being moved in a scene will not be instantly transferred to the requested location, but interpolated instead. This interpolation can sometimes be a bit slow to react, especially if the simulator thread is not able to keep up, which can cause sound sources to emit a short ghost sound (pops) when they're in a location where they should no longer be audible. Also, if a source is not included in a simulation for a while, and then later reintroduced, the simulation will begin interpolating from an old position to the latest, again causing ghost audio.
Simulating AI hearing: This also causes issues when trying to emulate AI hearing with SteamAudio/Phonon. The simulations require a lot of processing power, but if you try to reduce this overhead by running the AI hearing simulations only intermittently, the simulation interpolation causes the simulations to be less accurate the less often you run them. Sometimes this can cause reflections to cascade, when moving from a high reverb/echo location to a more silent one, where the reflections will affect the simulation results long after the listener has moved away from the reverberation, or the source has been removed from the listeners "reach".
Resetting all the objects included in the simulation seems to alleviate this interpolation, but this instead causes the simulated objects to interpolate from location "zero" to the target position.
The optimal scenario for AI hearing would be to run simulations only when needed (ie. when the player causes an audio disturbance), and applying the simulation results to the least amount of samples. Direct simulations would only require a single float (the noise amount) to achieve results. Reverb might need more samples (obviously?), since you can't achieve reverberation without the tail, or at least a buffer for the tail, but the interpolation issue still remains.
Conclusion: Disabling the interpolation entirely for selected simulator, or being able to forcefully reset listener/source to a specific location immediately, should negate these issues.
This seems to be related to the following issues: [C API] IPLDirectEffect fades in from silence #224 Delay in updating the position for calculating reflections. #212