This library is absolutely impossible to debug!

[misyltoad]

Please, please, please consider open sourcing this library.

It is absolutely impossible to debug anything with this library and I really feel like it does not value my time as a developer when I try to integrate it.

Other audio solutions, at least offer the option of commercial support, or symbols or whatever, but Steam Audio gives you absolutely nothing. I am not saying the way forward is to do that though, it would really be ideal if it was just open unlike the other audio solutions.

Doing anything with Steam Audio means I spend a good 20% of my time writing something I think should work and then I validate it, and everything looks fine and then 80% of it trying to figure out why my application just crashes with what appear to be completely valid inputs or the output buffer is left unchanged.

There is no logging. You can add a log callback, but it is never called. Ever, not once have I ever had it tell me anything. Everything just fails silently or crashes, it is so completely and utterly frustrating!

Just yesterday, I tried to implement ambisonic soundscapes in my engine. iplAmbisonicsDecodeEffectApply does absolutely nothing when I call it, and also doesn't have any error state/logging. It doesn't even touch the dst buffer. All the data I am passing in seems valid and correct so why is it just doing nothing? Clearly I am doing something wrong, but finding out what is basically impossible!

I know of multiple open source projects and companies that have abandoned their attempt to implement this library because actually getting anything working is impossible. The only examples/samples are outdated Unity/Unreal integrations which isn't clear to follow or doesn't implement what I need.

If you want Steam Audio to actually be adopted by games and open source projects, then please just open source it. There is a lot of demand for an audio library with this feature set -- but right now it is simply not compelling for anyone due to how difficult it is to integrate.

[Slartibarty]

I feel exactly the same way, I'm working with a legacy game engine and made a push to rewrite the entire sound engine and replace hardcoded EAX effects with Steam Audio's HRTF and scene processing.

I got as far as implementing HRTF after a lot of frustration with crashes inside of the Steam Audio DLL, unfortunately I effectively shelved everything after that because I was unable to diagnose what I was doing wrong with the API. With enough determination I could eventually get things working, but it feels like I'm taking shots in the dark when passing data into the API, it frustrates me to no end.

I'm not even confident about the code I've written using the API that "works" right now, because I don't get any feedback about things that could be wrong, this wouldn't be so bad if the code was available, where I could check exactly what every function expects.

I've been considering OpenAL Soft's effects as an alternative because getting past garbage output and crashes I can't debug feels like trying to climb an infinitely high wall.

[lakulish]

@Joshua-Ashton Could you provide more details on the issue you're facing with iplAmbisonicsDecodeEffectApply? If you have a sample piece of code that doesn't work, I can take a look and diagnose what's going on.

@Slartibarty Could you provide more details on what you were trying to do that was crashing or not working with Steam Audio?

That said, your point about better diagnostics is well taken, and we will work to improve this over the next few releases.

[IainWinter]

I've also been fighting over the last few days to get this library up and running with FMOD in C++. I ran into some issues with the docs vs the actual implementation.

In the User Guide's Fmod section for occlusion it says Set SIMULATION_OUTPUTS to the address of the IPLSimulationOutputs structure.

but looking at the code here https://github.com/ValveSoftware/steam-audio/blob/master/fmod/src/spatialize_effect.cpp#L1061 it seems to be expecting a pointer to the IPLSource, not IPLSimulationOutputs.

Is this correct?

After I figured that out, I am now attempting to add reverb, but if I call iplFMODSetReverbSource with a seemingly normal IPLSource:

IPLSourceSettings sourceSettings = {};
sourceSettings.flags = IPL_SIMULATIONFLAGS_REFLECTIONS;

iplSourceCreate(m_simulator, &sourceSettings, &m_listenerSource);
iplSourceAdd(m_listenerSource, m_simulator);

iplFMODSetReverbSource(m_listenerSource);

there is a mysterious access violation on 0xFFF... inside of phonon.dll after a few frames

With the user guide only being 2 steps, it's unclear what the steps are to fix this issue, or where the configuration has gone wrong.

I really like the results from this library when it works, but I have to agree that the only way to fix issues it to take shots in the dark.

edit:

after sometime looking at the steam_reverb.cpp file, I found that this access violation happens when it tries to call iplReflectionEffectApply. If I use visual studio to decrement the channel count in the reflectionParams by 1 it doesn't error but I see no place to specify this count differently as it's calculated from the maxOrder. I feel like I am misunderstanding something here...

[misyltoad]

Could you provide more details on the issue you're facing with iplAmbisonicsDecodeEffectApply? If you have a sample piece of code that doesn't work, I can take a look and diagnose what's going on.

I was doing a first order ambisonic, so order is 1 (i also tried 0, it is not clear whether 0 or 1 should be first order), there are 4 channels. The input buffer has 4 channels of 512 samples, and the output buffer is 2 channels of 512 samples.

I validated that all input data was in range 0-1 with no INF/NaN.

I hardcoded the orientation for now just to get things off the ground, I also tried with the exact orientation axes provided in phonon.h but it didn't change anything.

{
        IPLAmbisonicsDecodeEffectSettings effectSettings
        {
            .speakerLayout
            {
                .type = IPL_SPEAKERLAYOUTTYPE_STEREO,
            },
            .hrtf = nullptr,
            .maxOrder = 2
        };

        iplAmbisonicsDecodeEffectCreate( g_iplContext, &audioSettings, &effectSettings, &job.ambisonicDecodeEffect );
}

...

{
    IPLCoordinateSpace3 orientation
    {
        .right = { 0, 1, 0 },
        .up = { 0, 0, 1 },
        .ahead = { 1, 0, 0 },
        .origin = { 0, 0, 0 },
    };

    IPLAmbisonicsDecodeEffectParams params
    {
        .order = 1,
        .hrtf = nullptr,
        .orientation = orientation,
        .binaural = IPL_TRUE,
    };

    iplAmbisonicsDecodeEffectApply( job.ambisonicDecodeEffect, &params, inputBuffer, outputBuffer );
}

[lakulish]

@Joshua-Ashton Looks like the error here is that you're setting params.binaural to IPL_TRUE, but setting params.hrtf and effectSettings.hrtf to nullptr. You need a valid HRTF in order to use binaural rendering.

One way to fix it would be to set params.binaural to IPL_FALSE, which will switch to panning instead of binaural rendering.

The other way would be to create an IPLHRTF object using iplHRTFCreate, and pass it in effectSettings.hrtf and params.hrtf.

Let me know if, after trying either of the above options, you still run into issues.

[lakulish]

@IainWinter Do you have a standalone piece of code that can reproduce this issue? I want to understand a little more about how you're setting up the effect, what the maxOrder is, and why the channel count calculation seems to be going wrong.

[misyltoad]

@lakulish I will try it with panning instead of binaural, thanks.

Again though, nothing in the documentation or API or logging or... tells me about this that I can tell. :(

Would be really nice if this was open source so I could just debug this myself instead of spending hours playing around.

[misyltoad]

Thank you, that seemed to fix my issue and I managed to get ambisonics working in my engine. :D

One other thing I am confused about is the affect of IPLCoordinateSpace3::origin in IPLAmbisonicsDecodeEffectParams::orientation.

I looked at the code for Unity/FMOD, etc and it just passes the raw player origin here but there is no source coordinate system... I don't understand what origin is meant to do with an ambisonic either. Is this value just unused?

[IainWinter]

@IainWinter Do you have a standalone piece of code that can reproduce this issue? I want to understand a little more about how you're setting up the effect, what the maxOrder is, and why the channel count calculation seems to be going wrong.

Thanks for looking at this

I've done my best to isolate the problem down to just fmod/phonon code. The dlls made this too big to store in an attachment, so I just created a public repo with it already built and a VS sln.

There are two branches, one uses prebuilt dlls, and the other builds them itself. I thought maybe there was a mismatch with the versions of the dlls or something, but it results in the same issue.

Both contain just a single file src/main.cpp that causes the issue.

You can access it here: https://github.com/IainWinter/Standalone-SteamAudio-Error

[voxelizedworld]

+1 for open-sourcing it.

I feel the same way. At first, we were stoked to use Steam Audio for our title, but when we decided to ship on GDK/Xbox, we had to ditch it and redo our audio pipeline. It's not supported on that platform and there's no commercial support, so it was a big headache. And to make things worse, when we were integrating it was near impossible to figure out what's going wrong because everything fails silently.

[lakulish]

@IainWinter Looks like the root cause of the crash is that the samplingRate value in IPLAudioSettings is 44.1 kHz, but (on my system at least) FMOD defaults to 48 kHz.

The Steam Audio FMOD Studio plugin uses FMOD's actual sampling rate in its calculations, but when the code uses the Steam Audio API to create IPLSource objects, the buffers are allocated using the wrong sampling rate. The IPLAudioSettings structure should contain the actual values of sampling rate and frame size used by the audio engine.

Changing line 104 to:

audioSettings.samplingRate = 48000;

should fix the issue. That said, I agree that Steam Audio could do a better job of handling and reporting situations like these.

[IainWinter]

Replying to https://github.com/ValveSoftware/steam-audio/issues/243#issuecomment-1449165948

I would have never guessed that! Such a small detail in all of the code. I switched it to 48k and it worked immediately. Thank you for your time, I can finally continue with my project :)

Iain

[saturnian-tides]

+1 for open-sourcing if possible, which I feel is in the spirit of Valve's big projects recently (like proton).

[mastercoms]

Fixed :) https://steamcommunity.com/games/596420/announcements/detail/7745698166044243233

[Xottab-DUTY]

This issue relates to #2. But it's not a complete duplicate of that. This issue highlights important topic: problems with the debugging during Steam Audio integration.

Since the library was released to open source now, we can freely close #2, but maybe we should discuss how we can improve the library in terms of the debugging in this exact issue and postpone closing it? Nevermind, version 4.5.2 now has working IPL_CONTEXTFLAGS_ENABLEVALIDATION flag and it's now possible to debug the library even without the need of compiling it because we have sources and debug symbols.

[IainWinter]

yay

the integration debugging was actually ok from what I remember in fmod/c++ because there are only like 3/6 files, it was just they all went into a black box eventually