How to use dart ffi gen with multiple dynamic libraries?

[thumbert]

Hi,

Me again (sorry!) I've posted this question on SO but no answers. I will post on SO what I learn from here.

In my attempt to create Dart bindings for GSL, I need to load two dynamic libraries libgsl.so and libgslcblas.so. I have used ffigen to generate the bindings and things work fine as long as all symbols called are from libgsl.so.

But there are issues when BLAS related functionality is needed (available from libgslcblas.so.) For example, there is a function gls_blas_dnrm2 which exists in libgsl.so that needs function cblas_dnrm2 from libgslcblas.so. So when I call the generated method for gsl_blas_dnrm2 (which gets resolved correctly) I get the error

/usr/lib/dart/bin/dart: symbol lookup error: /usr/local/lib/libgsl.so: undefined symbol: cblas_dnrm2

which makes sense as the symbol cblas_dnrm2 is in libgslcblas.so (I can confirm that.)

I've tried to modify the ffigen generated lookup to look in both libraries, something like this

  Gsl((ffi.DynamicLibrary, ffi.DynamicLibrary) dynamicLibraries) {
    ffi.Pointer<T> aux<T extends ffi.NativeType>(String symbolName) {
      print('looking up symbol $symbolName');
      if (dynamicLibraries.$1.providesSymbol(symbolName)) {
        return dynamicLibraries.$1.lookup(symbolName);
      } else if (dynamicLibraries.$2.providesSymbol(symbolName)) {
        return dynamicLibraries.$2.lookup(symbolName);
      } else {
        throw StateError('No symbol $symbolName');
      }
    }
    _lookup = aux;
  }

but that doesn't help because the _lookup function does not get called for the cblas_dnrm2 at all.

From C (example), I can simply do gcc -Wall -o example example.c -lgsl -lgslcblas -lm and things work.

Do you have any suggestions on how I can resolve this? I did not find this issue documented. Many thanks for your help!

T

[dcharkes]

I am not sure I fully understand the problem. Could you elaborate?

You mean that the native code in libgsl.so needs libgslcblas.so? In that case it is the C dynamic linker that will try to find libgslcblas.so. (It will kind of try to do dlopen+dlsym.) Is libgslcblas.so on your include path? Or you can try doing DynamicLibrary.open('path/to/libgslcblas.so') to preload it for the C linker to find.

/usr/lib/dart/bin/dart: symbol lookup error: /usr/local/lib/libgsl.so: undefined symbol: cblas_dnrm2

I am a bit suprised we don't see libgslcblas.so in the error message. Is the way that libgsl.so is compiled not knowing that it needs libgslcblas.so as a dependency? Can you run ldd /usr/local/lib/libgsl.so to see what the dynamic dependencies are?

[thumbert]

Thanks for looking at this.

Even if I load libgslcblas.so explicitly, it does not work

  var lib = DynamicLibrary.open('/usr/local/lib/libgslcblas.so');
  print(lib.providesSymbol('cblas_dnrm2'));  // true
  var lib2 = DynamicLibrary.open('/usr/local/lib/libgsl.so');
  print(lib2.providesSymbol('gsl_blas_dnrm2'));  // true

  var x = [1.0, 2.0, 3.0];
  var v = gsl.gsl_vector_alloc(3);
  for (var i = 0; i < x.length; i++) {
    gsl.gsl_vector_set(v, i, x[i]);
  }

  /// line below throws symbol lookup error: /usr/local/lib/libgsl.so: undefined symbol: cblas_dnrm2
  gsl.gsl_blas_dnrm2(v);  
  gsl.gsl_vector_free(v);

On my C example file I get

$ ldd example
    linux-vdso.so.1 (0x00007ffe83ffa000)
    libgsl.so.27 => /usr/local/lib/libgsl.so.27 (0x00007f9b5b400000)
    libgslcblas.so.0 => /usr/local/lib/libgslcblas.so.0 (0x00007f9b5b6eb000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9b5b000000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9b5b319000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f9b5b749000)

Here's on libgsl

~/gsl_dart$ ldd /usr/local/lib/libgsl.so
        linux-vdso.so.1 (0x00007ffce4ff2000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f515eb19000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f515e800000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f515ef03000)

Thank you, T

[dcharkes]

So gsl.cblas_dnrm2(v) would fail because gsl is passed DynamicLibrary.open('/usr/local/lib/libgsl.so'). This could be solved by generating bindings for one cblas separately and importing those in your FFIgen config for gsl. But that's not the problem you're having.

On my C example file I get

$ ldd example
    linux-vdso.so.1 (0x00007ffe83ffa000)
    libgsl.so.27 => /usr/local/lib/libgsl.so.27 (0x00007f9b5b400000)
    libgslcblas.so.0 => /usr/local/lib/libgslcblas.so.0 (0x00007f9b5b6eb000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9b5b000000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9b5b319000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f9b5b749000)

So because you pass the linker flags, the C program is compiled in such a way that it loads both libgsl.so and libgslcblas.so at startup. And once gls_blas_dnrm2 is invoked, cblas_dnrm2 is already loaded into the program by the dynamic linker.

Here's on libgsl

~/gsl_dart$ ldd /usr/local/lib/libgsl.so
        linux-vdso.so.1 (0x00007ffce4ff2000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f515eb19000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f515e800000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f515ef03000)

DynamicLibrary.open('/usr/local/lib/libgsl.so') does not trigger loading libgslcblas.so because it's not in the dependency list. See documentation for ldd. I'm not sure why not, it would make sense if it was, because it is clearly required.

Maybe open an issue the repo of the library provider why the requirement does not show up in ldd?

As a workaround for the missing entry in ldd, you could try to compile an new dummy dynamic library that has both libgsl.so and libgslcblas.so in it's ldd required dynamic libraries list. And then DynamicLibrary.open that dylib in Dart, which should then trigger loading both libraries.

[thumbert]

I made it to work. I RTFM autoconf!

So I've added the lines below to the configure.ac file at line 131.

AC_CHECK_LIB([m],[cos])

# ------------------------------------------------------------------
# Check for BLAS
# ------------------------------------------------------------------
AC_CHECK_LIB([cblas],[cblas_dgemm],[],[
  echo "Library libcblas not found. Looking for GSL cblas." 
  echo -n " The present value of LDFLAGS is: " 
  echo $LDFLAGS
  AC_CHECK_LIB([gslcblas],[cblas_dgemm],[],[
    echo "Library libgslcblas not found. gsl_dart requires a cblas library." 
    echo "You may be required to add a cblas library to the LIBS "
    echo "environment variable. "
    echo ""
    echo -n " The present value of LDFLAGS is: " 
    echo $LDFLAGS
    echo ""
  ],[])
],[])

# ------------------------------------------------------------------
# Check for GSL library (must check for BLAS first)
# ------------------------------------------------------------------
AC_CHECK_LIB([gsl], [gsl_vector_get], [], 
    [echo ""
        echo "GSL not found."
        echo "gsl_dart requires the GSL library."
    echo ""
    echo -n " The present value of LDFLAGS is: " 
    echo $LDFLAGS
        exit -1
    ],[])

Then autoreconf --verbose --install --force.

Then the usual ./configure && make && make install did it.

ldd /usr/local/lib/libgsl.so
        linux-vdso.so.1 (0x00007ffe1f174000)
        libgslcblas.so.0 => /usr/local/lib/libgslcblas.so.0 (0x00007fcf95879000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fcf9577d000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcf95000000)

Thanks for the gentle help along the way.

T