Crash when loading external library

[xslittlegrass]

It seems that calling an external library in a library function leads to crash of mathematica kernals.

These are the steps to reproduce the crash:

File /tmp/external_lib/addone.c:

int addone(int x){
    return x+1;
}

Compile to dynamic library

gcc -Wall -fpic -shared -o libaddone.so addone.c

which generate file /tmp/external_lib/libaddone.so

Load into Mathematica

src = "
  #include <wll_interface.h>
  int addone(int x);
  int test(int x)
  {
    return addone(x);
  }
  DEFINE_WLL_FUNCTION(test)";

CCompilerDriver`$CCompiler = {"Compiler" -> 
   CCompilerDriver`GenericCCompiler`GenericCCompiler, 
  "CompilerInstallation" -> "/usr/bin", "CompilerName" -> "g++-7"}

mylib = CreateLibrary[src, "wll_test", 
  "IncludeDirectories" -> {"/home/mengxiwu/.Mathematica/include"}, 
  "LibraryDirectories" -> {"/tmp/external_lib/"}, 
  "CompileOptions" -> "-std=c++17 -fPIC", 
  "LinkerOptions" -> "-laddone"]

test = LibraryFunctionLoad[mylib, "wll_test", {Integer}, Integer]
test[3]

Any ideas for the crash? Thanks.

[njpipeorgan]

addone(int) is dynamically linked here, which means that libaddone.so should be able to found. Add the path of the dynamic library to LD_LIBRARY_PATH would solve the problem:

SetEnvironment["LD_LIBRARY_PATH"->Environment["LD_LIBRARY_PATH"]<>":/tmp/external_lib/"]

I also notice that addone is compiled by gcc. To avoid name mangling problem, I would declare addone as extern "C" int addone(int);

[xslittlegrass]

I've changed to use g++ to compile addone, but now the library doesn't seem to load. To simplify the issue, both addone and wll_test are compiled using g++

g++-7 -Wall -fpic -shared -o libaddone.so addone.cc
g++-7 -shared -o wll_test.so -std=c++17 -fPIC -I/usr/local/Wolfram/Mathematica/12.0/SystemFiles/IncludeFiles/C -I/home/xslittlegrass/.Mathematica/include wll_test.cc -laddone -L/tmp/external_lib/

Load in Mathematica:

SetEnvironment[
 "LD_LIBRARY_PATH" -> 
  Environment["LD_LIBRARY_PATH"] <> ":/tmp/external_lib/"]

test = LibraryFunctionLoad["/tmp/external_lib/wll_test.so", 
"wll_test", {Integer}, Integer]

gives error

LibraryFunction::libload: 
The function wll_test was not loaded from the file /tmp/external_lib/wll_test.so.

Any idea about this error? Also, Mathematica doesn't seem to give more details about the causes of not able to load the library, how would you debug this type of loading error in general? Thanks.

[njpipeorgan]

I am able to reproduce the error but cannot find anything wrong here. My suspicion is that it should be something related to LD_LIBRARY_PATH. An alternative would be to link addone statically, otherwise I have no solution to the problem as for now.

[xslittlegrass]

Thanks for the reply.

In my real usage case, addone is a dynamic library that I have no control of, how can I create a shared library wll_test.so that is statically linked to addone so that it can be loaded by mathematica?

[njpipeorgan]

I have found a solution--using -rpath to hard code the search path. Here is what I did:

Compile addone to shared library

$ g++ -Wall -fPIC -shared -o libaddone.so addone.cpp

Compile wll_test using CreateLibrary

src = "
    #include <wll_interface.h>
    int addone(int x);
    int test(int x)
    {
        return addone(x);
    }
    DEFINE_WLL_FUNCTION(test)
";
mylib=CreateLibrary[src, "wll_test",
  "IncludeDirectories"->{"<path>/wll-interface/include/"},
  "LibraryDirectories"->{"/tmp/external_lib/"},
  "CompileOptions"->"-std=c++17 -fPIC",
  "LinkerOptions"->"-laddone -Wl,-rpath=/tmp/external_lib"]

Load the library

test=LibraryFunctionLoad[mylib,"wll_test",{Integer},Integer]

[xslittlegrass]

It works for me now. Thanks.

FYI, I found that it also works if we set the environment variable in the same terminal before launching Mathematica:

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/tmp/external_lib/
$ Mathematica&

[njpipeorgan]

So it is indeed a problem with LD_LIBRARY_PATH, interesting. Thanks!