[C Api] v4.1.3 ReflectionEffect cannot run correctly and the output is silent (test case)

[comeonlby]

According to the documentation, ReflectionEffect is called，but it cannot run correctly and the output result is silent. (Tested with Steam Audio 4.1.3, MacOS 10.15.7 Intel, Xcode 11.3.1) Is this a bug or am I using the API incorrectly?

Click to expand

``` #include #include #include #include #include "phonon.h" #include "WavFile.h" int main(int argc, char** argv) { argv[1] = (char *)"speech1.wav"; argv[2] = (char *)"out_speech1_Reflection_Effect.wav"; WavInFile *fileReader = new WavInFile(argv[1]); if(fileReader ==NULL){ printf("in file is null \n"); } int sampleRate = fileReader->getSampleRate(); int numBits = fileReader->getNumBits(); int channels = fileReader->getNumChannels(); WavOutFile *fileWriter = new WavOutFile(argv[2], sampleRate, numBits, 4); int framesize = 1024; float *fInBuffer = (float *)malloc(sizeof(float)*(framesize * channels)); float *fOutBuffer = (float *)malloc(sizeof(float)*(framesize * 4)); IPLerror error; // Context IPLContextSettings contextSettings{}; contextSettings.version = STEAMAUDIO_VERSION; IPLContext context{}; error = iplContextCreate(&contextSettings, &context); if(error!=IPL_STATUS_SUCCESS) { printf("error=%d\n",error); return 0; } // Audio IPLAudioSettings audioSettings{}; audioSettings.samplingRate = sampleRate; audioSettings.frameSize = 1024; // the size of audio buffers we intend to process // Scene IPLSceneSettings sceneSettings{}; sceneSettings.type = IPL_SCENETYPE_DEFAULT; //IPL_SCENETYPE_EMBREE IPLScene scene = nullptr; error = iplSceneCreate(context, &sceneSettings, &scene); if(error!=IPL_STATUS_SUCCESS) { printf("error=%d\n",error); return 0; } // four vertices of a unit square in the x-y plane IPLVector3 vertices[4] = { {0.0f, 0.0f, 0.0f}, {1.0f, 0.0f, 0.0f}, {1.0f, 1.0f, 0.0f}, {0.0f, 1.0f, 0.0f} }; // triangle indices use counter-clockwise winding order IPLTriangle triangles[2] = { {0, 1, 2}, {0, 2, 3} }; IPLMaterial materials[1] = { { {0.1f, 0.1f, 0.1f}, 0.5f, {0.2f, 0.2f, 0.2f} } }; // both triangles use the same material IPLint32 materialIndices[2] = {0, 0}; IPLStaticMeshSettings staticMeshSettings{}; staticMeshSettings.numVertices = 4; staticMeshSettings.numTriangles = 2; staticMeshSettings.numMaterials = 1; staticMeshSettings.vertices = vertices; staticMeshSettings.triangles = triangles; staticMeshSettings.materialIndices = materialIndices; staticMeshSettings.materials = materials; IPLStaticMesh staticMesh = nullptr; error = iplStaticMeshCreate(scene, &staticMeshSettings, &staticMesh); if(error!=IPL_STATUS_SUCCESS) { printf("error=%d\n",error); return 0; } iplStaticMeshAdd(staticMesh, scene); iplSceneCommit(scene); // Simulation IPLSimulationSettings simulationSettings{}; simulationSettings.flags = IPL_SIMULATIONFLAGS_REFLECTIONS; // this enables reflection simulation simulationSettings.sceneType = IPL_SCENETYPE_DEFAULT; simulationSettings.reflectionType = IPL_REFLECTIONEFFECTTYPE_CONVOLUTION; // see below simulationSettings.maxNumRays = 4096; simulationSettings.numDiffuseSamples = 32; simulationSettings.maxDuration = 2.0f; simulationSettings.maxOrder = 1; simulationSettings.maxNumSources = 8; simulationSettings.numThreads = 1; simulationSettings.samplingRate = audioSettings.samplingRate; simulationSettings.frameSize = audioSettings.frameSize; IPLSimulator simulator = nullptr; error = iplSimulatorCreate(context, &simulationSettings, &simulator); if(error!=IPL_STATUS_SUCCESS) { printf("error=%d\n",error); return 0; } iplSimulatorSetScene(simulator, scene); iplSimulatorCommit(simulator); // Source IPLSourceSettings sourceSettings{}; sourceSettings.flags = IPL_SIMULATIONFLAGS_REFLECTIONS; // this enables reflections simulator for this source IPLSource source = nullptr; error = iplSourceCreate(simulator, &sourceSettings, &source); if(error!=IPL_STATUS_SUCCESS) { printf("error=%d\n",error); return 0; } iplSourceAdd(source, simulator); iplSimulatorCommit(simulator); // SimulationInputs IPLCoordinateSpace3 sourceCoordinates{.right = IPLVector3{1, 0, 0}, .up = IPLVector3{0, 1, 0}, .ahead = IPLVector3{0, 0, 1}, .origin = IPLVector3{1, 1, 1}}; // the world-space position and orientation of the source IPLCoordinateSpace3 listenerCoordinates{.right = IPLVector3{1, 0, 0}, .up = IPLVector3{0, 1, 0}, .ahead = IPLVector3{0, 0, -1}, .origin = IPLVector3{0, 0, 0}}; // the world-space position and orientation of the listener IPLSimulationInputs inputs{}; inputs.flags = IPL_SIMULATIONFLAGS_REFLECTIONS; inputs.source = sourceCoordinates; iplSourceSetInputs(source, IPL_SIMULATIONFLAGS_REFLECTIONS, &inputs); IPLSimulationSharedInputs sharedInputs{}; sharedInputs.listener = listenerCoordinates; sharedInputs.numRays = 4096; sharedInputs.numBounces = 16; sharedInputs.duration = 2.0f; sharedInputs.order = 1; sharedInputs.irradianceMinDistance = 1.0f; iplSimulatorSetSharedInputs(simulator, IPL_SIMULATIONFLAGS_REFLECTIONS, &sharedInputs); // typically run in a separate thread iplSimulatorRunReflections(simulator); // ReflectionEffect IPLReflectionEffectSettings ReffectSettings; ReffectSettings.type = IPL_REFLECTIONEFFECTTYPE_CONVOLUTION; ReffectSettings.irSize = 96000; // 2.0f (IR duration) * 48000 (sampling rate) ReffectSettings.numChannels = 4; IPLReflectionEffect Reffect = nullptr; error = iplReflectionEffectCreate(context, &audioSettings, &ReffectSettings, &Reffect); if(error!=IPL_STATUS_SUCCESS) { printf("error=%d\n",error); return 0; } // ReflectionMixer IPLReflectionMixer Mixer = nullptr; error = iplReflectionMixerCreate(context, &audioSettings, &ReffectSettings, &Mixer); if(error!=IPL_STATUS_SUCCESS) { printf("error=%d\n",error); return 0; } IPLAudioBuffer inBuffer; iplAudioBufferAllocate(context, channels, audioSettings.frameSize, &inBuffer); IPLAudioBuffer outBuffer; iplAudioBufferAllocate(context, 4, audioSettings.frameSize, &outBuffer); // Process int frame_count = 0; while (!fileReader->eof()) { frame_count ++; printf("frame_count=%d\n",frame_count); int realSampleCnt = fileReader->read(fInBuffer, framesize * channels); int sample_per_channel = realSampleCnt/channels; iplAudioBufferDeinterleave(context, fInBuffer, &inBuffer); // typically run in the main update thread IPLSimulationOutputs outputs{}; iplSourceGetOutputs(source, IPL_SIMULATIONFLAGS_REFLECTIONS, &outputs); IPLReflectionEffectParams Rparams = outputs.reflections; // this can be passed to a reflection effect (see below) Rparams.type = simulationSettings.reflectionType; Rparams.ir = outputs.reflections.ir; Rparams.irSize = 96000; Rparams.numChannels = 4; iplReflectionEffectApply(Reffect, &Rparams, &inBuffer, &outBuffer, nullptr); iplReflectionMixerApply(Mixer, &Rparams, &outBuffer); iplAudioBufferInterleave(context, &outBuffer, fOutBuffer); fileWriter->write(fOutBuffer, sample_per_channel*4); } // Release iplAudioBufferFree(context, &outBuffer); iplReflectionEffectRelease(&Reffect); iplReflectionMixerRelease(&Mixer); iplContextRelease(&context); iplSceneRelease(&scene); iplSourceRelease(&source); iplSimulatorRelease(&simulator); delete fileReader; delete fileWriter; return 0; } ```

[lakulish]

From what I can tell, this code tries to get mixed reflections from an IPLReflectionMixer object, but doesn't send audio to it from iplReflectionEffectApply. Try one of the following:

• Pass Mixer when calling iplReflectionEffectApply: iplReflectionEffectApply(Reffect, &Rparams, &inBuffer, &outBuffer, &Mixer);, or
• Comment out the call to iplReflectionMixerApply. This will directly render the output of the IPLReflectionEffect.

Let me know if neither of these resolves the issue.

[comeonlby]

From what I can tell, this code tries to get mixed reflections from an IPLReflectionMixer object, but doesn't send audio to it from iplReflectionEffectApply. Try one of the following:

• Pass Mixer when calling iplReflectionEffectApply: iplReflectionEffectApply(Reffect, &Rparams, &inBuffer, &outBuffer, &Mixer);, or
• Comment out the call to iplReflectionMixerApply. This will directly render the output of the IPLReflectionEffect.

Let me know if neither of these resolves the issue.

Thank you for your patient answer. I tried both of the methods you provided, but the final output is still silent.I suspect there is a problem with the generation of IPLReflectionEffectIR, but I don't have any log prompts.

[lakulish]

I took a closer look at the code, and it looks like the impulse response is silent because the listener is at one of the vertices of the geometry (0, 0, 0). Try moving the listener slightly away from the geometry (e.g. by adding an offset in the z-axis). That, in addition to one of the fixes I suggested above, should hopefully resolve the issue. Let us know if it doesn't.

[comeonlby]

I took a closer look at the code, and it looks like the impulse response is silent because the listener is at one of the vertices of the geometry (0, 0, 0). Try moving the listener slightly away from the geometry (e.g. by adding an offset in the z-axis). That, in addition to one of the fixes I suggested above, should hopefully resolve the issue. Let us know if it doesn't.

You're right. I tried to move the listener slightly away from the geometry and it worked. Thank you again for your patient answer, which is of great help to me. Best wishes.