Sound Not Passing Through Audio Mixer & Can't Find The File in Unity Plugin.

[mike22199]

Hello,

I encountered an issue while using the rnbo unity plugin. I created an RNBO project as shown in the image below, but after building it into a plugin, it doesn't work. When I place it in the Audio Mixer, the sound doesn't pass through. However, when I export the same file using the RNBO Web feature, it works fine.

Here are my speculations on the possible causes:

Could it be due to the inability to read the buffer file? I used the copy sample feature during export and copied the generated Media file into the export folder.

Are there any RNBO functions I used that might be unsupported, such as FFT?

I'm using Max 8.6.2 and RNBO

[mike22199]

I have resolved the previous issue as I had not set the index to the same number. I apologize for overlooking this detail.

I now have another question: I am currently developing the plugin on an Android system. The RNBO Plugin works perfectly on the Windows system, but it does not work on the Android system. I would like to ask if I need to use the NDK mentioned in the documentation for export in order to make it work.

Additionally, when I tried to use the NDK for export, I encountered issues when assigning the DCMAKE_TOOLCHAIN_FILE. The problems I encountered are as described below:

C:\Users\User\Desktop\EchoFrame\rnbo.unity.audioplugin-main (1)\rnbo.unity.audioplugin-main\build_android>cmake .. -DANDROID_PLATFORM=29 -DANDROID_ABI=arm64-v8a -DCMAKE_TOOLCHAIN_FILE="C:\Program Files\Unity\Hub\Editor\2021.3.10f1\Editor\Data\PlaybackEngines\AndroidPlayer\NDK\build\cmake\android.toolchain.cmake" -DRNBO_EXPORT_DIR="Z:\Documents\export\sync-cycle" -DRNBO_CPP_DIR="Z:\Documents\export\sync-cycle\rnbo" -DPLUGIN_NAME="Sync Cycle" -DANDROID_CPP_FEATURES=exceptions -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - failed -- Check for working CXX compiler: C:/Program Files/Unity/Hub/Editor/2021.3.10f1/Editor/Data/PlaybackEngines/AndroidPlayer/NDK/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe -- Check for working CXX compiler: C:/Program Files/Unity/Hub/Editor/2021.3.10f1/Editor/Data/PlaybackEngines/AndroidPlayer/NDK/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe - broken CMake Error at D:/Cmake/share/cmake-3.29/Modules/CMakeTestCXXCompiler.cmake:60 (message): The C++ compiler

"C:/Program Files/Unity/Hub/Editor/2021.3.10f1/Editor/Data/PlaybackEngines/AndroidPlayer/NDK/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: 'C:/Users/User/Desktop/EchoFrame/rnbo.unity.audioplugin-main (1)/rnbo.unity.audioplugin-main/build_android/CMakeFiles/CMakeScratch/TryCompile-wle3we'

Run Build Command(s): "C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe" cmTC_9189c.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /v:n
.NET Framework ??MSBuild ? 17.10.4+10fbfbf2e
撌脩???撱箇蔭??2024/6/3 銝? 02:02:25??

蝭€暺?1 (?身?格?) 銝?撠? "C:\Users\User\Desktop\EchoFrame\rnbo.unity.audioplugin-main (1)\rnbo.unity.audioplugin-main\build_android\CMakeFiles\CMakeScratch\TryCompile-wle3we\cmTC_9189c.vcxproj"??
SetBuildDefaultEnvironmentVariables:
  ANDROID_HOME=C:\Program Files (x86)\Android\android-sdk
  ANDROID_SDK_ROOT=C:\Program Files (x86)\Android\android-sdk
  ANT_HOME=
  JAVA_HOME=C:\Program Files\Android\jdk\jdk-8.0.302.8-hotspot\jdk8u302-b08
  NDK_ROOT=C:\\Microsoft\AndroidNDK\android-ndk-r23c
PrepareForBuild:
  甇?撱箇??桅? "cmTC_9189c.dir\Debug\"??
  甇?撱箇??桅? "C:\Users\User\Desktop\EchoFrame\rnbo.unity.audioplugin-main (1)\rnbo.unity.audioplugin-main\build_android\CMakeFiles\CMakeScratch\TryCompile-wle3we\Debug\"??
  甇?撱箇??桅? "cmTC_9189c.dir\Debug\cmTC_9189c.tlog\"??
InitializeBuildStatus:
  ?望??鈭?"AlwaysCreate"嚗?甇斗迤?典遣蝡?"cmTC_9189c.dir\Debug\cmTC_9189c.tlog\unsuccessfulbuild"??
  甇?霈???唾? "cmTC_9189c.dir\Debug\cmTC_9189c.tlog\unsuccessfulbuild"??
ClCompile:
  C:\\Microsoft\AndroidNDK\android-ndk-r23c\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe -c -fdiagnostics-format=msvc --gcc-toolchain="C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\x86_64-4.9\\prebuilt\\windows-x86_64" -target "x86_64-none-linux-android1" --sysroot="C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot" -isystem "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\include\\x86_64-linux-android" -isystem "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\lib64\\clang\\12.0.9\\include" -isystem "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\android\\support\\include" -isystem "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\cxx-stl\\llvm-libc++\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\cxx-stl\\llvm-libc++abi\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\include\\x86_64-linux-android" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\android\\support\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\lib64\\clang\\12.0.9\\include" -gline-tables-only -o "cmTC_9189c.dir\\Debug\\testCXXCompiler.o" -Wall -O3 -fno-strict-aliasing -fomit-frame-pointer -fno-exceptions -ffunction-sections -fdata-sections -fstack-protector -fpic -fno-short-enums -fno-rtti -std=c++11 -D __ANDROID_MIN_SDK_VERSION__=1 -D NDEBUG -D ANDROID -D _FORTIFY_SOURCE=2 -D "CMAKE_INTDIR=\"Debug\"" -x c++  -g -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wformat -Werror=format-security -fexceptions -O0 -fno-limit-debug-info "C:\\Users\\User\\Desktop\\EchoFrame\\rnbo.unity.audioplugin-main (1)\\rnbo.unity.audioplugin-main\\build_android\\CMakeFiles\\CMakeScratch\\TryCompile-wle3we\\testCXXCompiler.cxx"
  testCXXCompiler.cxx
Link:
  C:\\Microsoft\AndroidNDK\android-ndk-r23c\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe -fdiagnostics-format=msvc -o"cmTC_9189c.dir\\Debug\\cmTC_9189c." -Wl,-soname="cmTC_9189c." -Wl,-rpath-link="C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\lib\\x86_64-linux-android" -Wl,-rpath-link="C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\lib\\x86_64-linux-android\\1" -Wl,-L"C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\lib\\x86_64-linux-android" -Wl,-L"C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\lib\\x86_64-linux-android\\1" -Wl,-L"C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\x86_64-4.9\\prebuilt\\windows-x86_64\\lib\\gcc\\x86_64-linux-android\\4.9.x" -Wl,-L"C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\cxx-stl\\llvm-libc++\\libs\\x86_64" -Wl,--no-undefined --gcc-toolchain="C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\x86_64-4.9\\prebuilt\\windows-x86_64" -target "x86_64-none-linux-android1" --sysroot="C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot" -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -shared  -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libgcc_real.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -Wl,--gc-sections cmTC_9189c.dir\\Debug\\testCXXCompiler.o "-latomic" "-lm" -lc++_static -lc++abi -llog -landroid
  ld: error: cannot open crtbegin_so.o: No such file or directory
  ld: error: unable to find library -llog
  ld: error: unable to find library -landroid
  ld: error: cannot open crtend_so.o: No such file or directory
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Application Type\Android\3.0\Android.Common.targets(125,5): error MSB6006: "clang.exe" 隞亥??Ⅳ 1 蝯???[C:\Users\User\Desktop\EchoFrame\rnbo.unity.audioplugin-main (1)\rnbo.unity.audioplugin-main\build_android\CMakeFiles\CMakeScratch\TryCompile-wle3we\cmTC_9189c.vcxproj]
撠? "C:\Users\User\Desktop\EchoFrame\rnbo.unity.audioplugin-main (1)\rnbo.unity.audioplugin-main\build_android\CMakeFiles\CMakeScratch\TryCompile-wle3we\cmTC_9189c.vcxproj" (?身?格?) 撱箇蔭摰? -- 憭望???

撱箇蔭憭望???

"C:\Users\User\Desktop\EchoFrame\rnbo.unity.audioplugin-main (1)\rnbo.unity.audioplugin-main\build_android\CMakeFiles\CMakeScratch\TryCompile-wle3we\cmTC_9189c.vcxproj" (?身?格?) (1) ->
(Link ?格?) ->
  C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Application Type\Android\3.0\Android.Common.targets(125,5): error MSB6006: "clang.exe" 隞亥??Ⅳ 1 蝯???[C:\Users\User\Desktop\EchoFrame\rnbo.unity.audioplugin-main (1)\rnbo.unity.audioplugin-main\build_android\CMakeFiles\CMakeScratch\TryCompile-wle3we\cmTC_9189c.vcxproj]

    0 ?郎??
    1 ?隤?

蝬??? 00:00:01.09

CMake will not be able to correctly generate this project. Call Stack (most recent call first): CMakeLists.txt:6 (project)

-- Configuring incomplete, errors occurred!

Sorry for the silly question, and thank you if there's any help.

[mike22199]

Hello,

I have resolved the previous issue by using the NDK files from Unity 2022. It seems that the NDK files in the Unity 2020 LTS version had some issues. I have successfully built the .so file.

However, I have encountered another issue. I would like to ask how to use the RNBO file built in the .so file as an Audio Mixer. I am unable to see the plugin in the Unity editor, so I cannot directly assign the Audio Mixer's Master to any Audio Source's Output.

Is it possible to assign it directly through C# code?

[jinpavg]

Hi @mike22199 , thank you for reaching out here. I'm glad you were able to resolve your previous issues.

I'm not sure what you are describing in this latest comment -- could you give a little more context, or share any screenshots or example code? Feel free to let us know more about what you are trying to do and how you'd like to use RNBO to do it.

[mike22199]

Here is your translated response:

Hi Alex @jinpavg ,

Thank you for your reply.

I am currently developing a MaxMSP FFT splitting tool and constructing it into Unity using RNBO for mobile development. I aim to have this plugin work through Unity's Audio Mixer. When playing sound through Unity, I want to control the plugin's gate to determine which frequency bands can pass through.

Currently, I have exported the project as a .so file using the NDK and imported this package into Unity. However, I cannot find the effect in the Audio Mixer-Effect's Add dropdown menu, unlike when exporting as a .dll file. I'm not sure if this issue is caused by the error mentioned below, or if files created using NDK cannot be edited and viewed through the Unity Editor.

Therefore, I am trying to use a C# script to assign the RNBO Plugin I created into the Output field of the AudioSource in a GameObject. I hope I have explained this clearly. I will include some images below to illustrate the issue, along with my files, to help you understand my problem better.

Thank you very much for your help.

Here's the all file (I have two test rnbo plugin called My Custom TFF Plugin and NATIVEMobileFFT)(these two plugin is build without NDK tool, so it can work and find in Audio Mixer Effect) https://drive.google.com/drive/folders/1W1vNW6bRNdjwQdqDtzM9mhElFVU6m8bL?usp=sharing

[jinpavg]

Hi @mike22199 , thank you for this additional context, I think that I understand the issue now.

I see that you are making an .so version of your device, for Android. However, you can't find the plugin in the dropdown as an effect in the Audio Mixer. I believe that the issue is that you also need to also build a .dll (assuming you are on windows) and add that resulting /Windows folder to the MobileFFTPlugin/Assets/Plugins/ directory alongside the /Android folder so that the plugin is accessible on the platform you are using to build the unity project.

Try this out and let me know if it works for you.

[mike22199]

It means that I need to build a .dll file with the same name to use it in the Unity editor. Apologies for the late reply; I will try this in the next few days.

Additionally, I would like to know if this plugin is compatible with Max/MSP, as I intend to use it in my academic paper.

I would like to know if it is acceptable for me to include this plugin in my academic paper. I will also cite this website in my references. Please let me know if this is okay.

[mike22199]

I have encountered an issue where my plugin works perfectly in the Unity Editor and when exported as an .EXE file, but fails to function as expected when exported as an Android .APK file.

In my setup, I have created a .dll file and placed it in the Plugins folder. My RNBO file contains six gates, each capable of filtering sound frequencies via a buffer file. For example, when the first gate is opened, only low-frequency sounds can pass through.

I have attached a audiosource to a gameObject and set its output to an AudioMixer containing the plugins. Upon startup, all gates of the plugin are set to be closed using code. I use Unity buttons to control the opening and closing of these gates. This method works flawlessly in the Unity Editor and in the .EXE build.

However, when I export the project as an Android .APK file, the program fails to control the plugin parameters as expected. Upon opening the application, sound plays immediately, as if the output settings were not read or applied. I have tried using Debug.log to read the Plugin's parameter values for debugging, but this method does not seem to work for reading the parameter values of the plugin.

Could you please help me identify what might be causing this issue? Is there any specific configuration needed for Android builds to ensure the plugin parameters are read and controlled correctly?

Thank you for your assistance.

[jinpavg]

Hi @mike22199 , thank you for this report. would it be possible for you to share a simple project that includes your plugin, with steps of how to set the param in the editor and then how to reproduce the problem in the android build?

[mike22199]

@jinpavg , Thanks for the reply, Here's my steps and project.

RNBO File Setup

My RNBO file includes 6 parameters and 6 buffers. I used the *~ function to create a filter that affects playback content based on the buffer I provided. The gate is opened and filtered through parameter control.

Exporting and Building the Plugin

I followed the process on your GitHub, exporting my RNBO file and building it into a .dll file named MobileFFTPlugin.dll. Then, I used this .dll file in my Unity project, intending to use it as an audio effect plugin.

Unity Setup

In Unity, I set up multiple buttons to control the gate switches in the RNBO file. Each switch represents a gate control for remainder values from 0 to 5.

Packaging the .SO File

Following your tutorial, I packaged the same RNBO file into a .SO file named libMobileFFTPlugin.so, and saved it in the same folder as MobileFFTPlugin.dll. However, I am not sure how to locate this .SO file in the code or Unity editor.

Here's the ZIP.file for my project, thank you again for this question. Drive: https://drive.google.com/file/d/1l2TfC8F-s3f4H2Hc2exO9UVzPNwDSeoU/view?usp=sharing

I also upload the Max/MSP file and buffer file here : https://drive.google.com/drive/folders/1RI4sbX4I_SChERIRd4h3ZDjiQ1mkPc60?usp=sharing

[jinpavg]

Hi @mike22199 , thank you -- we had to make an android device available to test this, and we will test and be back in touch. Thank you for your patience and sharing this information!

[jinpavg]

@mike22199 , I'm afraid that I can't open your .zip, I'm very sorry -- it looks like the file is corrupted,

However, I've been thinking about your issue and I have an idea for something you might try. I'm wondering if the settings you used when building the plugin (using our instructions) are mismatched with Player Settings in the Unity Editor.

To be specific, I wonder if you are targeting ARMv7 in Player Settings, while the plugin we build is meant for ARM64:

If you set the Minimum API level to Android 10.0 and then check ARM64 for the Target Architecture, and then build/deploy your project, does your plugin work as you expect?

If this does not help, could you try another way to share the project with me?

[jinpavg]

I'm closing this ticket for now, but certainly feel free to re-open if this is still a concern.