Capture does not start when using wwise audio

[ChadKeating]

Describe the bug Capture does not start when using wwise audio.

Observations

• It works fine when wwise is not set as the audio capture.
• Wwise.asmdef doesnt exist anymore. When I upgraded from 2021.1.0 to 2021.1.4 the Wwise.asmdef was removed. So I dont understand the instructions requiring it to be referenced? I can get the code to compile by assigning other wwise asmdefs individually i.e.
• I have also tried on the latest 2022 version or wwise.

Your Setup (please complete the following information):

• Unity version: 2020.3.30f1
• AVPro Movie Capture version: AVProMovieCapture-v5.1.4-Trial
• Wwise version: 2021.1.4.7707
• Operating system version: Windows 10 21H2 (OS Build 19044.2846)
• Capture component used: CaptureFromScreen
• Capture settings (resolution, frame-rate, codec):

Logs

[AVProMovieCapture] Start File Capture: 1920x1080 @ 60.00fps [RGBA32] vcodec:'H264' audio source:'Wwise' 48000hz 8 channels acodec:'AAC' to file: 'P:/Screen Recordings/Game\MovieCapture_2023-05-04_21-26-00_1920x1080.mp4'

[AVProMovieCapture] Failed to create recorder

[Chris-RH]

Did you add AVPRO_MOVIECAPTURE_WWISE_SUPPORT to Scriping Define Symbols? Have you included the CapturefromWWise component? Do you have a full error message?

[ChadKeating]

Hi Chris,

Yes the define was added. It shows a big error message in the inspector if its not.

I've tried adding CaptureFromWwise manually as well as letting the CaptureFromScreen script set it up for me. Neither worked.

The full stack trace for those message (which are the only ones that appear) are:

[AVProMovieCapture] Start File Capture: 1920x1080 @ 60.00fps [RGBA32] vcodec:'H264' audio source:'Wwise' 48000hz 8 channels acodec:'AAC' to file: 'P:/Screen Recordings/Game\MovieCapture_2023-05-05_16-08-14_1920x1080.mp4'
UnityEngine.Debug:Log (object)
RenderHeads.Media.AVProMovieCapture.CaptureBase:PrepareCapture () (at Assets/Plugins/RenderHeads/AVProMovieCapture/Runtime/Scripts/Internal/CaptureBase.cs:2338)
RenderHeads.Media.AVProMovieCapture.CaptureFromScreen:PrepareCapture () (at Assets/Plugins/RenderHeads/AVProMovieCapture/Runtime/Scripts/Components/CaptureFromScreen.cs:168)
RenderHeads.Media.AVProMovieCapture.CaptureBase:StartCapture () (at Assets/Plugins/RenderHeads/AVProMovieCapture/Runtime/Scripts/Internal/CaptureBase.cs:2531)
RenderHeads.Media.AVProMovieCapture.CaptureBase:Update () (at Assets/Plugins/RenderHeads/AVProMovieCapture/Runtime/Scripts/Internal/CaptureBase.cs:3001)

[AVProMovieCapture] Failed to create recorder
UnityEngine.Debug:LogError (object)
RenderHeads.Media.AVProMovieCapture.CaptureBase:PrepareCapture () (at Assets/Plugins/RenderHeads/AVProMovieCapture/Runtime/Scripts/Internal/CaptureBase.cs:2420)
RenderHeads.Media.AVProMovieCapture.CaptureFromScreen:PrepareCapture () (at Assets/Plugins/RenderHeads/AVProMovieCapture/Runtime/Scripts/Components/CaptureFromScreen.cs:168)
RenderHeads.Media.AVProMovieCapture.CaptureBase:StartCapture () (at Assets/Plugins/RenderHeads/AVProMovieCapture/Runtime/Scripts/Internal/CaptureBase.cs:2531)
RenderHeads.Media.AVProMovieCapture.CaptureBase:Update () (at Assets/Plugins/RenderHeads/AVProMovieCapture/Runtime/Scripts/Internal/CaptureBase.cs:3001)

Another observation is the sound from wwise after the error happens is very stuttery. I looked at the wise logs but this is all that happens.

We also have unity audio disabled just fyi.

[Chris-RH]

Can you reduce the number of channels to 2 and/or 6 and see if that works please? (It looks like AAC might be limited to 1, 2 or 6 - https://learn.microsoft.com/en-us/windows/win32/medfound/aac-encoder#input-types) Could you try a different audio codec perhaps?

[ChadKeating]

Hiya,

I figured out the issue and have made a work around for my case. The issue seems to be wwise uses the audio devices channel config when offline rendering. Regardless of whether the project is setup to be a stereo only project or the mainmix is set to be stereo, I even tried setting the channel count in the wwise unity settings but it always came out with 8 channels when it started recording. The wwise sdk docs mentioned it pulled the audio device channel config so I changed to a stereo setup and recording works fine.

Aside from using a setup audio device, I worked around this by hardcoding the number of channels I wanted from wwise and then unpacking the interleved samples.

The edited version of the CaptureAudioFromWwise script.

    void Update()
        {
            if (_isRendererRecording && _capture != null && _capture.IsCapturing() && !_capture.IsPaused())
            {
                var sampleCount = AkSoundEngine.UpdateCaptureSampleCount(_outputDeviceId);
                if (sampleCount <= 0)
                {
                    return;
                }

                var buffer = new float[sampleCount];
                var count = AkSoundEngine.GetCaptureSamples(_outputDeviceId, buffer, (uint)buffer.Length);
                if (count <= 0)
                {
                    return;
                }

                var totalChannelCount = ChannelCount;
                var channelsToKeep = ChannelCountClamped; //ChannelCountClamped is the config I added

                // Create a new buffer for keeping only two channels
                var stereoBuffer = new float[count / totalChannelCount * channelsToKeep];
                for (var i = 0; i < count / totalChannelCount; i++)
                {
                    for (var j = 0; j < channelsToKeep; j++)
                    {
                        stereoBuffer[i * channelsToKeep + j] = buffer[i * totalChannelCount + j];
                    }
                }

                _capture.EncodeAudio(stereoBuffer);
            }
        }

I havent tested this much but it got me to the next step. I can get a video output with clear sound now!

[Chris-RH]

ahh, that's very helpful, thank you :)