linking MWEngine with others libraries (libsndfile, libsamplerate)

[scar20]

As this issue may be unrelated to this forum, a more generic question here will be; how to link MWEngine with another library when you want to be able to access MWEngine at the c++ level to pass some values (here double buffers).

I actually did this (and find quite wasteful): Have a jni interface to get a double buffer from libsndfile (resampled from libsamplerate in the same call), then send that buffer as a jdouble buffer in java, then call JavaUtility.createSampleFromBuffer() which will then internally create another double buffer in c++. It will be better to just create that buffer in c++ in the jni and call SampleManager::setSample() directly from there without back and forth java to c++ to java to c++. The problem is I can't access MWEngine from that jni.

Then I though that I could link libsndfile (and libsamplerate) with MWEngine. So, in a new branch, I added libsndfile.so and libsamplerate.a in a lib directory and added some lines in the CMakeLists.txt: added a directory variable at the top of the file

#########################################
## experimental link with libsndfile and libsamplerate
set(LIB_DIR ${CMAKE_CURRENT_SOURCE_DIR}/src/main/libs)
#########################################

and those just before the wrapping directives

####################################
## Experimental link with external libsndfile and libsamplerate
####################################
add_library(sndfile SHARED IMPORTED)
set_target_properties(sndfile PROPERTIES IMPORTED_LOCATION
        ${LIB_DIR}/libsndfile/lib/${CMAKE_BUILD_TYPE}/${ANDROID_ABI}/libsndfile.so)

add_library(samplerate STATIC IMPORTED)
set_target_properties(samplerate PROPERTIES IMPORTED_LOCATION
        ${LIB_DIR}/libsamplerate/lib/${CMAKE_BUILD_TYPE}/${ANDROID_ABI}/libsamplerate.a)

target_include_directories(${target} PRIVATE
        ${LIB_DIR}/libsndfile/include
        ${LIB_DIR}/libsamplerate/include)

target_link_libraries(${target} sndfile samplerate)

But I got this error from the built: unknown target CPU 'intel'

Looking into more details seem to indicate that come from the "include" since it appear at the end of a -I command(I'm not well versed at all with CMake). Can I just remove the include directory target and just copy its content after the build? Anyhow, the question is what would be to "proper" way to do this and what are the pitfalls to avoid. More I read, less I am sure... libsndfile need to be shared because of license, but I can produce libsamplerate as static or shared, both libs in release or debug for all abi's used by MWEngine.

Below, the built log (MWEngine+libsndfile+libsamplerate) and further below, what I have actually tested (external lib not linked with MWEngine) with the call from java main and the jni call for completeness.

built log:

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':mwengine:externalNativeBuildDebug'.
> Build command failed.
  Error while executing process C:\Users\Sylvain\AppData\Local\Android\Sdk\cmake\3.10.2.4988404\bin\ninja.exe with arguments {-C C:\Users\Sylvain\AndroidStudioProjects\MWEngine\mwengine\.cxx\cmake\debug\x86_64 mwengine_wrapped}
  ninja: Entering directory `C:\Users\Sylvain\AndroidStudioProjects\MWEngine\mwengine\.cxx\cmake\debug\x86_64'
  [1/51] Building C object CMakeFiles/mwengine.dir/src/main/cpp/drivers/opensl_io.c.o
  FAILED: CMakeFiles/mwengine.dir/src/main/cpp/drivers/opensl_io.c.o 
  C:\Users\Sylvain\AppData\Local\Android\Sdk\ndk\23.1.7779620\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe --target=x86_64-none-linux-android21 --gcc-toolchain=C:/Users/Sylvain/AppData/Local/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/windows-x86_64 --sysroot=C:/Users/Sylvain/AppData/Local/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/windows-x86_64/sysroot  -I../../../../src/main/cpp -I../../../../src/main/libs/libsndfile/include -I../../../../src/main/libs/libsamplerate/include -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -O3 -march=x86-64 -msse4.2 -mpopcnt -m64 -mtune=intel -fno-limit-debug-info  -fPIC -MD -MT CMakeFiles/mwengine.dir/src/main/cpp/drivers/opensl_io.c.o -MF CMakeFiles\mwengine.dir\src\main\cpp\drivers\opensl_io.c.o.d -o CMakeFiles/mwengine.dir/src/main/cpp/drivers/opensl_io.c.o   -c ../../../../src/main/cpp/drivers/opensl_io.c
  error: unknown target CPU 'intel'
  note: valid target CPU values are: i386, i486, winchip-c6, winchip2, c3, i586, pentium, pentium-mmx, pentiumpro, i686, pentium2, pentium3, pentium3m, pentium-m, c3-2, yonah, pentium4, pentium4m, prescott, nocona, core2, penryn, bonnell, atom, silvermont, slm, goldmont, goldmont-plus, tremont, nehalem, corei7, westmere, sandybridge, corei7-avx, ivybridge, core-avx-i, haswell, core-avx2, broadwell, skylake, skylake-avx512, skx, cascadelake, cooperlake, cannonlake, icelake-client, icelake-server, tigerlake, sapphirerapids, alderlake, knl, knm, lakemont, k6, k6-2, k6-3, athlon, athlon-tbird, athlon-xp, athlon-mp, athlon-4, k8, athlon64, athlon-fx, opteron, k8-sse3, athlon64-sse3, opteron-sse3, amdfam10, barcelona, btver1, btver2, bdver1, bdver2, bdver3, bdver4, znver1, znver2, znver3, x86-64, geode
  [2/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/processors/baseprocessor.cpp.o
  [3/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/utilities/debug.cpp.o
  [4/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/utilities/bulkcacher.cpp.o
  [5/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/utilities/levelutility.cpp.o
  [6/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/utilities/samplemanager.cpp.o
  [7/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/utilities/bufferpool.cpp.o
  [8/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/utilities/wavewriter.cpp.o
  error: unknown target CPU 'intel'

  [9/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/utilities/bufferutility.cpp.o
  [10/51] Building CXX object CMakeFiles/mwengine.dir/src/main/cpp/utilities/diskwriter.cpp.o
  ninja: build stopped: subcommand failed.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':mwengine:externalNativeBuildRelease'.
> Build command failed.
  Error while executing process C:\Users\Sylvain\AppData\Local\Android\Sdk\cmake\3.10.2.4988404\bin\ninja.exe with arguments {-C C:\Users\Sylvain\AndroidStudioProjects\MWEngine\mwengine\.cxx\cmake\release\armeabi-v7a mwengine_wrapped}
  ninja: Entering directory `C:\Users\Sylvain\AndroidStudioProjects\MWEngine\mwengine\.cxx\cmake\release\armeabi-v7a'

  ninja: error: '../../../../src/main/libs/libsndfile/lib/Release/armeabi-v7a/libsndfile.so', needed by '../../../../build/intermediates/cmake/release/obj/armeabi-v7a/libmwengine_wrapped.so', missing and no known rule to make it

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 34s

The MainActivity call:

        FileHandler fileHandler = FileUtil.getFileHandler(filePath);
        JavaUtilities.createSampleFromBuffer("one", fileHandler.dbuffer.length,
                1, fileHandler.dbuffer, null);
        _sample.setSample(SampleManager.getSample("one"));

FileHandler is a java wrapper for the buffer and parameters:

protected class FileHandler {
        FileHandler() {}
        int format;
        int channels;
        int sampleRate;
        int length;
        int outputLength;
        double[] dbuffer;
    }

and the jni call:

extern "C"
JNIEXPORT jobject JNICALL
Java_com_scar_fileio_FileUtil_getFileHandler(JNIEnv *env, jclass clazz, jstring file_path) {
    const char* cpath = env->GetStringUTFChars(file_path, 0);
    SndfileHandle file;  // libsndfile handle
    file = SndfileHandle (cpath);
    int channels = file.channels();
    long frames = file.frames();
    long length = channels*frames;
    int samplerate = file.samplerate();
    int format = file.format();

    // read or convert
    int native = 48000; // hard coded native (output) sample rate
    float ibuf[length]; // input buffer from file frames * channels
    long olength = static_cast<long>(((double)native/(double)samplerate) * (double)length); // new output length
    float obuf[olength]; // output buffer with new output length
    // prepare output buffer
    double *buf = static_cast<double *>(malloc(sizeof(double) * olength));
    LOGD("!!! double buf created: %d\n", (int)olength);

    SRC_DATA src_data;  // anonymous struct to pass data to/from libsamplerate
    if (file.samplerate() != native) {
        LOGD("!!! File is *not* native sample rate !!!\n");
        const char* version = src_get_version();
        LOGD("LibSampleRate version : %s\n", version);
        // first read the file in a float buffer
        file.read(ibuf, length);
        src_data.data_in = ibuf;
        src_data.data_out = obuf;
        src_data.input_frames = (long)length;
        src_data.output_frames = (long)olength;
        src_data.src_ratio = double(native)/double(file.samplerate());  // output / input
        src_simple(&src_data, SRC_SINC_BEST_QUALITY, file.channels()); // samplerate conversion with best quality

        for (int i = 0; i < olength; i++) {
            buf[i] = static_cast<double >(obuf[i]);
        }
    } else {
        LOGD("!!! File is native sample rate !!!\n");
        file.read(buf, olength);
    }
    ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
    ///////// Here I will want to pass the buffer to SampleManager insead of creating all the stuff below ///////////
    ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

    jdoubleArray jdbuf = env->NewDoubleArray((int)olength);
    env->SetDoubleArrayRegion(jdbuf, 0, (int)olength, (jdouble *) buf);

    //Getting class objects is a class file that exists in dex and can be viewed by analyzing the apk tool
    jclass packagecls = env->FindClass("com/scar/fileio/MainActivity$FileHandler");
    //Get the method id of the constructor of this class and the function signature of the method
    jmethodID construcMethodID = env->GetMethodID(packagecls, "<init>", "()V");
    //Create this java object
    jobject packageobj = env->NewObject(packagecls, construcMethodID);

    //Properties of Operating Objects

    jfieldID intFormatID = env->GetFieldID(packagecls, "format", "I");
    jfieldID intChannelsID = env->GetFieldID(packagecls, "channels", "I");
    jfieldID intSampleRateID = env->GetFieldID(packagecls, "sampleRate", "I");
    jfieldID intLengthID = env->GetFieldID(packagecls, "length", "I");
    jfieldID intOutLengthID = env->GetFieldID(packagecls, "outputLength", "I");
    jfieldID doubleDBufferID = env->GetFieldID(packagecls, "dbuffer", "[D");

    env->SetIntField(packageobj, intFormatID, format);
    env->SetIntField(packageobj, intChannelsID, (int)channels);
    env->SetIntField(packageobj, intSampleRateID, file.samplerate());
    env->SetIntField(packageobj, intLengthID, (int)length);
    env->SetIntField(packageobj, intOutLengthID, (int)olength);

    env->SetObjectField(packageobj, doubleDBufferID, jdbuf);

    free(buf);
    return packageobj;
}

[igorski]

But I got this error from the built: unknown target CPU 'intel' Looking into more details seem to indicate that come from the "include" since it appear at the end of a -I command(I'm not well versed at all with CMake). Can I just remove the include directory target and just copy its content after the build?

Looks like the original library isn't Android specific (which is not a necessarily a problem) as the ABI for intel is known as x64. I think you don't need to include those directories at all as the .so-file would provide the library in its binary, compiled form which the linker can then resolve just fine (for instance similar to all libraries currently used by the default make file for MWEngine).

However I'm not sure what contents of those folders actually are 🙈

How was the .so built ? Was it built specifically for all Android ABI's listed for MWEngine / your project ?

[scar20]

About your last question; those two libs come clearly from a linux environment with added support for win and apple. The built is minded for one target as it will be usually the case on those platforms. But they also use CMake as an alternative to autoconf and make. So what I did is create Android Studio project using "hello libs" (from new->import sample:ndk) as a template and use the gen-lib part of it as my starting point. I put the repository copy of the libs there and I use MWEngine gradle file as the template for the ndk settings. So yes, it build for "armeabi-v7a", "arm64-v8a", "x86_64" and use the same cflags as the engine. I tried not to mess too much with their CMakeLists.txt although since they use include (GNUInstallDirs), I had to make some ajustments to "grab" the generated headers and output lib and put those at the right place in a "distribution" directory inside the Android Studio project. With this setup, I managed to get all the output libs in debug and release for all abi's. I could test that they worked called from JNI independently of MWEngine. So I can call them and get result from java.

For the main problem, the short story is it seem to originate in MWEngine CMakeLists.txt line 33 "-mtune=intel" flag.

Now, the long story: I find that I had another problem since the output libs were put in "RelWithDebInfo" folder (the actual name of the CMake configuration) and I have that error in the built: ninja: error: '../../../../src/main/libs/libsndfile/lib/Release/armeabi-v7a/libsndfile.so', needed by '../../../../build/intermediates/cmake/release/obj/armeabi-v7a/libmwengine_wrapped.so', missing and no known rule to make it

So I renamed the folder to "Release" and now it build release too, exept: the build stop at the x86_64 pass with the same error unknown target CPU 'intel', In build/intermediates/cmake/, libmwengine_wrapped.so is generated for arm64-v8a and armeabi-v7a in debug and release but fail for x86_64 so no aar output is produced.

I made a text search for "intel" in libsndfile and libsamplerate projects but that returned only some references in some .m4 files and we dont actually build the libs, just using it.

Doing the same in MWEngine project , I found a reference in CMakeLists.txt line 33:

if (${ANDROID_ABI} MATCHES "x86_64")
    SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -march=x86-64 -msse4.2 -mpopcnt -m64 -mtune=intel")
endif()

Google search with "CMAKE_C_FLAGS -mtune=intel" returned nothing. After others unsuccessful searches with "clang" in keyword, it give me this:

Specify bit size of immediate TLS offsets (AArch64 ELF only): 12 (for 4KB) | 24 (for 16MB, default) | 32 (for 4GB) | 48 (for 256TB, needs -mcmodel=large)

-mtune=<arg>

But they do not say what to put in <arg> but that leave me to suspect something more about Android Studio, so I commented out all the lines referring to the libs in CMakeLists.txt and try to :mwengine:assemble as usual; it fail at the same place! It look like there is a new feature in Android Studio (Bumblebee patch1, I don't dare to get patch2 yet) preventing you to use that flag.... Looking also at the build message, the suggested valid target cpu values seem very very specific: note: valid target CPU values are: i386, i486, winchip-c6, winchip2, c3, i586, pentium, pentium-mmx, pentiumpro, i686, pentium2, pentium3, pentium3m, pentium-m, c3-2, yonah, pentium4, pentium4m, prescott, nocona, core2, penryn, bonnell, atom, silvermont, slm, goldmont, goldmont-plus, tremont, nehalem, corei7, westmere, sandybridge, corei7-avx, ivybridge, core-avx-i, haswell, core-avx2, broadwell, skylake, skylake-avx512, skx, cascadelake, cooperlake, cannonlake, icelake-client, icelake-server, tigerlake, sapphirerapids, alderlake, knl, knm, lakemont, k6, k6-2, k6-3, athlon, athlon-tbird, athlon-xp, athlon-mp, athlon-4, k8, athlon64, athlon-fx, opteron, k8-sse3, athlon64-sse3, opteron-sse3, amdfam10, barcelona, btver1, btver2, bdver1, bdver2, bdver3, bdver4, znver1, znver2, znver3, x86-64, geode I don't know what to make of this... And I don't want to f*** up the CMakeLists.txt by putting I don't know what there or by removing the flag altogether. Any ideas?

[scar20]

I think I got it. I spotted x86-64 in the list of valid cpu's and it looked as a good candidate (that is what we build for). So I replaced the "intel" value in the flag with this one. SET(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -march=x86-64 -msse4.2 -mpopcnt -m64 -mtune=x86-64") and it built well and produced the aar's. So I uncommented the lines for the externals libs and it built well too.

Eureka? I'll have to test if I can access the libs from within MWEngine and also provide some JNI's to access them independently, ie: provide a method that take a path and produce a result that will be passed internally to MWEngine without returning to java (or perhaps returning a boolean just to know everything is OK).

[scar20]

Well, after what seems successful build, the produced aar's let me access my external native method in java without complaining but when I try to run the app, call to those methods will crash with "FATAL EXCEPTION No implementation found for \<method>" as if the jni-native.cpp do not get included in the aar's. I tried both SHARED and STATIC for the jni built with same results.

Here is what I tried lastly: addition to MWEngine CMakeLists.txt:

#########################################
## experimental link with libsndfile and libsamplerate
set(LIB_DIR ${CMAKE_CURRENT_SOURCE_DIR}/src/main/ext/libs)
set(EXT_SRC ${CMAKE_CURRENT_SOURCE_DIR}/src/main/ext)
set(EXT_JNI_SRC ${EXT_SRC}/jni/fileio.cpp)
#########################################

and just before the Swig section:

####################################
## Experimental link with external libsndfile and libsamplerate
####################################
add_library(sndfile SHARED IMPORTED)
set_target_properties(sndfile PROPERTIES IMPORTED_LOCATION
        ${LIB_DIR}/libsndfile/lib/${CMAKE_BUILD_TYPE}/${ANDROID_ABI}/libsndfile.so)

add_library(samplerate STATIC IMPORTED)
set_target_properties(samplerate PROPERTIES IMPORTED_LOCATION
        ${LIB_DIR}/libsamplerate/lib/${CMAKE_BUILD_TYPE}/${ANDROID_ABI}/libsamplerate.a)

add_library(fileio STATIC ${EXT_JNI_SRC})

target_include_directories(fileio PRIVATE
        ${LIB_DIR}/libsndfile/include
        ${LIB_DIR}/libsamplerate/include)

target_link_libraries(${target} sndfile samplerate fileio)

FileIO.java just contain 4 methods:

public class FileIO {

    // JNI method

    public native static void setNativeSampleRate(int sampleRate);

    public native static short [] getShortBufferFromFile(String path);

    public native static void installFilesFromAssets(String inputPath, String outputPath);

    public native static boolean createSampleFromFileExternal(String key, String path);

}

and fileio.cpp contain the implementation code for those methods; it have the headers for libsndfile, libsamplerate and MWEngine.

installFilesFromAssets open files to read from asset with libsndfile, if not native sample rate, convert them with libsamplerate and open new files to write in user space with libsndfile.

createSampleFromFileExternal open a file to read with libsndfile and read direcly into a MWEngine::AudioBuffer and pass it to MWEngine::SampleManager::setSample.

Any ideas why the app using the produced aar's cant find the implementation code? It is possible that you cannot mix swig and jni build together? Maybe I do it in the wrong way... again.. And tell me if this begin to be too much out of subject.

[scar20]

I'm putting this aside for now. I have sufficient access to libsndfile and libsamplerate to be used in parallel with MWEngine at install time but later on I will need a more internal way to make those libs to "speak" to MWEngine at the c++ level - to open a flac file with libsndfile and pass the resulting buffer as an AudioBuffer to MWEngine::SampleManager::setSample. Closing this thread for now - some progress but unsuccessful result.