[Unity] Turning Indirect sound/Reflections on completely breaks the audio.

[JagHip]

For some reason, every time Reflections is enabled the audio glitches.

If i use the realtime option with more than one audio source, the audio stutters a lot. And if i use the baked audio source option, the audio starts really quiet, until you leave the radius, at which point the audio becomes very loud, and pops every time you enter or leave the source radius.

This video shows exactly what happens:

https://youtu.be/qJgrsgNiziY

Please, tell me if i'm doing something wrong.

So, does anyone knows what is causing this, and how to fix it?

[lakulish]

For the real-time option, can you turn on the stats overlay in the Game tab and see what the audio DSP usage looks like?

For the baked option, that definitely looks like unexpected behavior. Can you provide some more information about your Steam Audio settings? Ideally, if you could provide a minimal project that reproduces the issue, that would be best.

[JagHip]

This is my original project: https://user-images.githubusercontent.com/76782925/103931865-be07d900-50ff-11eb-9014-ba2d1991cf6c.mp4

and although the DSP load is high in this case, is not as bad as in my second project, that i created when trying to reproduce the problem: https://user-images.githubusercontent.com/76782925/103932259-5736ef80-5100-11eb-8407-994bc53dc5d8.mp4

Then i tried to use the baked option and something weird happened. When i use the Uniform Floor placement strategy, it does this: so every probe point sticks to the walls. And if i use the Centroid placement, the reflections only starts working after i enter the source radius, but it won't "turn off" after i leave the radius...

I'm starting to suspect that something is wrong with my Unity version (2019.3.5f1), so i'll try to update it and see if it changes anything.

[Edit] After updating Unity, i actually was able to use the Uniform Floor placement strategy. Now i'll try to reproduce the baked reflections issue, and post the project here when i'm done.

[JagHip]

Okay, it's been a while, and i'll just give up now... This problem really is an anomaly that apparently, only affects this specific project, because every time i try to reproduce it on another project, everything just works fine, which is a good thing. I'll keep the discussion open, just in case, but for now that's all. Thanks for replying!

[lakulish]

It's certainly odd that the issue can't be reproduced in a new project, and that updating Unity changed the behavior of the Uniform Floor placement. I agree, I'll keep the issue open for the time being. Thanks!

[nicholas-maltbie]

Hello, I know this issue may be pretty old but I just ran into this as well when attempting to utilize steam audio. Each audio source with Realtime reflections enabled increases DPS load by about 20-30% meaning after 4 are added the audio quality will degrade significantly.

In order to investigate the issue, I made a fresh project and was able to reproduce the issue https://github.com/nicholas-maltbie/TestSteamAudio2

I'm not sure if this is an issue with the default settings, my sound file, the version of unity I'm using (2020.3.5f1) or something else entirely. I can investigate more settings if needed as well. After about 3-4 Realtime reflection sources have been added, the DPS load is maxed out. Is there a way to decrease the quality of the simulation enough to allow for more Realtime reflection sources without overloading resources? Is this the intended behavior, it seemed that baked sources also accounted for a significant amount of DPS Load.

I can help look into the issue more if there are any other resources or permutations that would be able to try related to the issue.

[achandak]

Hi @nicholas-maltbie - Unfortunately, I could not repro the issue with the project you shared. I am seeing 20-25% DSP usage with about 11 sources in your project. Could you provide your system specs? I used Unity version 2020.1.6 - so don't think version change would make any difference.

[nicholas-maltbie]

Hi @achandak, thanks for the update.

I'm using a Microsoft Windows 10 Laptop with the following specs:

• Processor - Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, 2208 Mhz, 6 Core(s), 12 Logical Processor(s)
• Installed Physical Memory (RAM) - 16.0 GB
• Display Name - NVIDIA GeForce RTX 2060 (1920 NVIDIA Cores, 6 GB GDDR6 Memory)

The exact model is a Predator PT515-51

I tried running the program in Unity version 2020.1.6 and I still ran into the error so version doesn't make a difference on my system. It may just be that my CPU and system specs are not sufficient to simulate more than 3-4 realtime sources.

I think this is caused by my CPU or performance being overwhelmed so I decided to check the steam audio settings and this is the low default that it was set to. I'm not sure why the DPS Load% is making out so quickly. Is there a windows driver or dependency that Steam Audio requires that does not come with windows by default?

The settings does seem to cast a lot of realtime rays but when adjusting the settings around a bit it doesn't seem to make a difference. After adding about 3-4 audio sources the DPS Load exceeds 100% and the program starts to fail.

It looked like the TrueNext audio supported realtime raycasts for GPU. This might be able to mitigate the issue but I haven't been able to get it working as of yet (Unity editor crashes, might have to setup another new project). Is there a setting I could use to significantly lower the resource usage of the simulation so it would be able to work on any system even if it is a lower end qualityof hardware? As of right now I just have realtime reflections disabled in my project but it would be nice to allow a user to enable them for a more immersive.

[lakulish]

We looked at this again, and we are able to reproduce the issue. Adjusting the number of rays or bounces won't affect the DSP usage, because the ray tracing work happens in a separate thread. Instead, try reducing the Ambisonics order from 1 to 0. That should give you a 4x reduction in DSP usage, but it does mean that the convolution reverbs won't have directionality.

Convolution is a CPU-intensive operation: 4 sources at 1st order Ambisonics means we're running 16 convolution reverbs. Having said that, we're looking into whether this might be a performance regression, and will update this issue when we know more.

Thanks!

[nicholas-maltbie]

Hey @lakulish thanks for the suggestion, your suggestion really helped :) I thought something like raycasts in a different thread might be happening because that was potentially a lot of raycasts.

The Ambisonics order from 1 to 0 significantly improved the performance by about 4x as you suggested. While experimenting with the other settings today, I also found out that the Duration parameter can be quite significant as well. Decreasing the duration from 1 second to 0.1 second decreased the DPS load significantly (even with ambisonics set at 1). While looking at the documentation, I found an interesting quote

https://valvesoftware.github.io/steam-audio/doc/phonon_unity.html#custom-preset-for-simulation-settings

Duration. This is the length of the impulse responses to generate. Increasing this improves the quality of the simulation, but beyond a certain point (depending on the number of sound sources), may result in audio glitching.

I don't know if this is the issue I'm running into but it might be related since adjusting duration helped significantly improve the performance of the project. In my game, I want to use these realtime reflections to allow a player to hear footsteps of other players around corners (but not through walls). Seeing as footstep sounds and sound effects are fairly short, having a shorter duration isn't a significant problem.

A nice feature request may be to allow for different sound effects to have different reflection times so I can lower the reflection duration significantly for short sound effects (which the player might not care about). Although if the reflection time is baked into some significant structure of the optimization, this may not be possible. This audio plugin is great and has a lot of fun features. I'm happy to help experiment if you need further tests or anything else.

[TforTrevor]

I am also having this issue where turning on reflections breaks the sound quality, however my DSP usage is only at 10% and I only have a single sound source. Turning off HRTF on the reflections slightly mitigates the issue. The issue occurs at the same extremity whether using baked or realtime reflections. When using both pathfinding and reflections (whether baked or realtime), the issue becomes significantly worse.

[lakulish]

@TforTrevor Do you have a video of the issue or a minimal Unity project that reproduces the issue? Thanks!