FFI: Issue locating symbols defined in Linux Shared Libraries

[nikeokoronkwo]

Overview

This issue concerns the ffi package and library and is about the dynamic library lookup functionality. Apparently, I was working on a project, and it failed to work on Linux because the ffi tools were not able to find the functions that ffigen had created ffi bindings for. Just to confirm the issue wasn't from the library, I ran nm -gD <library-file> and noticed that the function was indeed in the dynamic library file.

This issue is currently being tracked in the project repository: nima-code/seagort#1

Steps to Reproduce

To build this project, you would need to have the rust set of tools, primarily cargo, to generate the shared library. You can as well run the Seagort Test Run on Github Actions here.

With a fresh clone of the repository at https://github.com/nima-code/seagort,

1. Build the engine at utils/engine using cargo

   cargo build --release

   If you would want to regenerate the ffigen bindings for the header file, you can regenerate the header file using [cbindgen]() (to install: cargo install --force cbindgen)

   cbindgen --config cbindgen.toml --lang c --output libseagort.h

2. Copy the generated library (at target/release) into the bridge/linux directory.
3. Run the example code.

   dart example/main.dart

Expected Outcome

The example code runs and the program exits successfully

Observed Outcome

The code doesn't run and the following error message is provided.

Unhandled exception:
Invalid argument(s): Couldn't resolve native function 'compile_js' in 'package:seagort/src/native/library.dart' : No asset with id 'package:seagort/src/native/library.dart' found. No available native assets. Attempted to fallback to process lookup. dart: undefined symbol: compile_js.

#0      Native._ffi_resolver.#ffiClosure0 (dart:ffi-patch/ffi_patch.dart)
#1      Native._ffi_resolver_function (dart:ffi-patch/ffi_patch.dart:1386:20)
#2      compile_js (package:seagort/src/native/library.dart)
#3      _compileJS (package:seagort/src/lib/compile.dart:9:18)
#4      compileJavaSript (package:seagort/src/lib/compile.dart:17:39)
#5      compileJS (package:seagort/src/js.dart:4:32)
#6      main (file:///home/runner/work/seagort/seagort/example/main.dart:4:3)
#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#8      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

Hence, the given bug.

If this issue isn't fixed or addressed, the project won't be able to be used or run for Linux users.

Additional Information

As mentioned before, you can run the following command (or whatever shared-library analysis tool you would want to use) and notice that the compile_js symbol is located there.

nm -gD bridge/linux/libseagort.so

[dcharkes]

Hi @nikeokoronkwo!

No asset with id 'package:seagort/src/native/library.dart' found.

To use @Native external functions, you need to tell Dart where the dynamic library is. You'll need to write a hook/build.dart that outputs a NativeCodeAsset with DynamicLoadingBundled with the path to the file. dart run and dart build will then know where to find the dynamic library at runtime.

See https://github.com/dart-lang/native/tree/main/pkgs/native_assets_cli for more documentation.

[nikeokoronkwo]

Hey @dcharkes

I haven't been able to get it work. Here's what I've come up with so far...

import 'dart:io';

import 'package:native_assets_cli/native_assets_cli.dart';
import 'package:native_toolchain_c/native_toolchain_c.dart';

const assetName = 'libseagort.so';
final packageAssetPath = Uri.file('bridge/linux/$assetName');

void main(List<String> args) async {
  await build(args, (config, output) async {
    final packageName = config.packageName;
    final assetPath = config.outputDirectory.resolve(assetName);
    final assetSourcePath = config.packageRoot.resolveUri(packageAssetPath);
    await File.fromUri(assetSourcePath).copy(assetPath.toFilePath());

     output.addDependencies([
       assetSourcePath,
       config.packageRoot.resolve('hook/build.dart'),
     ]);

    output.addAsset(
      NativeCodeAsset(
        package: packageName, 
        name: assetName, 
        file: assetPath,
        linkMode: DynamicLoadingBundled(), 
        os: config.targetOS
      )
    );
  });
}

Some guidance would be appreciated

[dcharkes]

That looks mostly like I expect to.

I think the architecture is missing in the NativeCodeAsset. Try:

      NativeCodeAsset(
        package: packageName, 
        name: assetName, 
        file: assetPath,
        linkMode: DynamicLoadingBundled(), 
        os: config.targetOS,
        architecture: config.targetArchitecture,
      )

[nikeokoronkwo]

Sadly, I'm still getting the same error message. I ran dart run --enable-experiment=native-assets example/main.dart on the code.

[mkustermann]

const assetName = 'libseagort.so';

main(...) {
    ...
    output.addAsset(
      NativeCodeAsset(
        package: packageName, 
        name: assetName, 
        file: assetPath,
        linkMode: DynamicLoadingBundled(), 
        os: config.targetOS
      ));
  ...
}

IIUC this hook/build.dart script would emit an asset with id package:seagort/libseagort.so. But the error message says the following:

No asset with id 'package:seagort/src/native/library.dart' found.

So it uses a different asset id (namely the default id, which is the package uri of the library containing @Native external declarations).

If you don't use NativeCodeAsset(..., name: 'src/native/library.dart) then you have to specify the different asset id in the library that uses @Native:

@DefaultAsset('package:seagort/libseagort.so')
library x;

@Native<...>(...)
external ... compile_js(...);

or use

@Native<...>(..., assetId: 'package:seagort/libseagort.so')
external ... compile_js(...);

Could that be the issue?

[nikeokoronkwo]

The problem persists, but now says that it can't find "asset with id 'package:seagort/libseagort.so' ". I am publishing a second branch on the github page to see what I've done concerning this

[dcharkes]

@nikeokoronkwo, no DynamicLibrary.open is needed:

String _compileJS(String data) {
  // final _ = readLibrary(); // Don't invoke DynamicLibrary.open
  final nativeData = data.toNativeUtf8().cast<Char>();
  final result = compile_js(nativeData);
  return result.cast<Utf8>().toDartString();
}

Then update hook/build.dart to point to the existing file on my machine:

const assetName = 'libseagort.so';
final packageAssetPath =
    Uri.file('utils/engine/target/release/libseagort.dylib');

Then running the tests works:

$ dart --enable-experiment=native-assets test
00:00 +0: test/seagort_test.dart: JS Tests Results                                                                                                                                                                                                                            Hello World
00:00 +0 -1: test/seagort_test.dart: JS Tests Results [E]                                                                                                                                                                                                                     
  Expected: 'Hello World'
    Actual: 'null'
     Which: is different.
            Expected: Hello Worl ...
              Actual: null
                      ^
             Differ at offset 0

Side note: I'd probably not use the @DefaultAsset in the Dart file, but change the asset ID in build.dart. (But this is just my preference.)

    output.addAsset(NativeCodeAsset(
      package: packageName,
      name: 'src/native/library.dart', // together with package this creates package:seagort/src/native/library.dart, which is the Dart library uri this asset is used in.
      file: assetPath,
      linkMode: DynamicLoadingBundled(),
      os: config.targetOS,
      architecture: config.targetArchitecture,
    ));

[nikeokoronkwo]

I sadly still haven't been able to get it to work. It displays the same error message...

[dcharkes]

I sadly still haven't been able to get it to work. It displays the same error message...

Try making a clean checkout just to make sure you're not hitting some caching issue with the native assets build.

Is the problem linux-specific? (It works for me on MacOS.)

[nikeokoronkwo]

Is the problem linux-specific? (It works for me on MacOS.)

Yes the problem is specifically for Linux. It works on other platforms (MacOS and Windows) with or without the use of hook/build.dart. (highlighted in the title)

[dcharkes]

dart --enable-experiment=native-assets test

I am confused. I have checked out your code (703554a992c3b07a45b5c08a76f6a00f08f45a5d) on a Linux machine and it works as expected.

~/src/prouesse/seagort/utils/engine$ cargo build --release
    Updating crates.io index
[...]

~/src/prouesse/seagort$ dart pub get
Resolving dependencies... (1.9s)
Downloading packages... (2.9s)
[...]

~/src/prouesse/seagort$ dart --enable-experiment=native-assets test
Building package executable... (5.6s)
Built test:test.
00:01 +0: test/seagort_test.dart: JS Tests Results                                                                                                                                Hello World
00:01 +0 -1: test/seagort_test.dart: JS Tests Results [E]                                                                                                                         
  Expected: 'Hello World'
    Actual: 'null'
     Which: is different.
            Expected: Hello Worl ...
              Actual: null
                      ^
             Differ at offset 0

  package:matcher              expect
  test/seagort_test.dart 12:7  main.<fn>.<fn>

To run this test again: dart test test/seagort_test.dart -p vm --plain-name 'JS Tests Results'
00:01 +0 -1: Some tests failed.                                                                                                                                                   

Consider enabling the flag chain-stack-traces to

The code works fine.

Could you try deleting .dart_tool (cached files), and running pub get and then dart --enable-experiment=native-assets test.

[nikeokoronkwo]

Have you tried running the example? example/main.dart

~/sg/utils/engine# cargo build --release
   Compiling proc-macro2 v1.0.78
   Compiling libc v0.2.153
[...]
    Finished release [optimized] target(s) in 24.84s

~/sg/utils/engine# cd ../..

~/sg# dart pub get
Resolving dependencies... 
[...]
Changed 54 dependencies!

~/sg# dart --enable-experiment=native-assets example/main.dart
Unhandled exception:
Invalid argument(s): Couldn't resolve native function 'compile_js' in 'package:seagort/libseagort.so' : No asset with id 'package:seagort/libseagort.so' found. No available native assets. Attempted to fallback to process lookup. dart: undefined symbol: compile_js.

#0      Native._ffi_resolver.#ffiClosure0 (dart:ffi-patch/ffi_patch.dart)
#1      Native._ffi_resolver_function (dart:ffi-patch/ffi_patch.dart:1386:20)
#2      compile_js (package:seagort/src/native/library.dart)
#3      _compileJS (package:seagort/src/lib/compile.dart:9:18)
#4      compileJavaSript (package:seagort/src/lib/compile.dart:17:39)
#5      compileJS (package:seagort/src/js.dart:4:32)
#6      main (file:///sg/example/main.dart:4:3)
#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#8      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

[dcharkes]

Ah, I missed that. Try dart --enable-experiment=native-assets run example/main.dart.

dacoharkes@dacoharkes-linux:~/src/prouesse/seagort$ rm -rf .dart_tool/
dacoharkes@dacoharkes-linux:~/src/prouesse/seagort$ dart pub get
Resolving dependencies... 
Downloading packages... 
Got dependencies!
dacoharkes@dacoharkes-linux:~/src/prouesse/seagort$ dart --enable-experiment=native-assets example/main.dart
Unhandled exception:
Invalid argument(s): Couldn't resolve native function 'compile_js' in 'package:seagort/libseagort.so' : No asset with id 'package:seagort/libseagort.so' found. No available native assets. Attempted to fallback to process lookup. dart: undefined symbol: compile_js.

#0      Native._ffi_resolver.#ffiClosure0 (dart:ffi-patch/ffi_patch.dart)
#1      Native._ffi_resolver_function (dart:ffi-patch/ffi_patch.dart:1523:20)
#2      compile_js (package:seagort/src/native/library.dart)
#3      _compileJS (package:seagort/src/lib/compile.dart:10:18)
#4      compileJavaSript (package:seagort/src/lib/compile.dart:18:39)
#5      compileJS (package:seagort/src/js.dart:4:32)
#6      main (file:///usr/local/google/home/dacoharkes/src/prouesse/seagort/example/main.dart:4:3)
#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#8      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)
dacoharkes@dacoharkes-linux:~/src/prouesse/seagort$ dart --enable-experiment=native-assets run example/main.dart
dacoharkes@dacoharkes-linux:~/src/prouesse/seagort$ dart --enable-experiment=native-assets example/main.dart

Background info: dart some/dart/script.dart only runs the Dart VM. dart run ... and dart test ... run the native assets build. So before running a dart script standalone two steps need to happen dart pub get and a way to trigger native asset builds. We currently don't have a command to trigger native asset builds (https://github.com/dart-lang/sdk/issues/52992).

(Both the error message for not having run pub and not having run native asset builds are not very informative.)

[nikeokoronkwo]

Thanks for that! Seems that was what was needed after all.

Before I close this, I wanted to ask a few things: Why is Linux specifically the environment having this issue and needing such workaround? Would I also need to implement this for other platforms (they are to be run on Github Actions by the way), even though they work without it?

[nikeokoronkwo]

Also I believe there should be a native_assets.yaml file generated. Here's what mine looked like:

# Native assets mapping for host OS in JIT mode.
# Generated by dartdev and package:native.
format-version:
  - 1
  - 0
  - 0
native-assets: {}

This was when testing it on a Mac.

[dcharkes]

Why is Linux specifically the environment having this issue and needing such workaround?

It's the same on all operating systems. You might have run dart test or dart run on your Mac.

dacoharkes-macbookpro2:seagort dacoharkes$ dart --enable-experiment=native-assets example/main.dart 
dacoharkes-macbookpro2:seagort dacoharkes$ rm -rf .dart_tool/ && dart pub get
Resolving dependencies... 
Downloading packages... 
Got dependencies!
dacoharkes-macbookpro2:seagort dacoharkes$ dart --enable-experiment=native-assets example/main.dart 
Unhandled exception:
Invalid argument(s): Couldn't resolve native function 'compile_js' in 'package:seagort/src/native/library.dart' : No asset with id 'package:seagort/src/native/library.dart' found. No available native assets. Attempted to fallback to process lookup. dlsym(RTLD_DEFAULT, compile_js): symbol not found.

#0      Native._ffi_resolver.#ffiClosure0 (dart:ffi-patch/ffi_patch.dart)
#1      Native._ffi_resolver_function (dart:ffi-patch/ffi_patch.dart:1523:20)
#2      compile_js (package:seagort/src/native/library.dart)
#3      _compileJS (package:seagort/src/lib/compile.dart:9:18)
#4      compileJavaSript (package:seagort/src/lib/compile.dart:17:39)
#5      compileJS (package:seagort/src/js.dart:4:32)
#6      main (file:///Users/dacoharkes/src/prouesse/seagort/example/main.dart:4:3)
#7      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#8      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)
dacoharkes-macbookpro2:seagort dacoharkes$ dart --enable-experiment=native-assets run example/main.dart 
dacoharkes-macbookpro2:seagort dacoharkes$ dart --enable-experiment=native-assets example/main.dart 
dacoharkes-macbookpro2:seagort dacoharkes$

The native_assets.yaml should contain an entry for package:seagort/src/native/library.dart:

$ cat .dart_tool/native_assets.yaml 
# Native assets mapping for host OS in JIT mode.
# Generated by dartdev and package:native_assets_builder.
format-version:
  - 1
  - 0
  - 0
native-assets:
  macos_arm64:
    package:seagort/src/native/library.dart:
      - absolute
      - /Users/dacoharkes/src/prouesse/seagort/.dart_tool/native_assets_builder/fe04e2fce1584a19ce171d039b9ab3e1/out/libseagort.so

[nikeokoronkwo]

Understood! Thanks for the fix