search

dart-archive / ffi

Utilities for working with Foreign Function Interface (FFI) code
https://pub.dev/packages/ffi
BSD 3-Clause "New" or "Revised" License
155 stars 32 forks source link

IOS Static Library "Failed to lookup symbol" #153

Closed jvkolyadich closed 2 years ago

jvkolyadich commented 2 years ago

I'm getting the error message

Failed to lookup symbol 'NDIlib_version': dlsym(RTLD_DEFAULT, NDIlib_version): symbol not found

which is the same one as in issue dart-lang/native#897, but I decided to open a new issue because

  1. The error message shows up in debug mode.
  2. The solution in issue 41 doesn't fix the problem.

Here's what I've done so far:

  1. Started a new project with the plugin_ffi template and followed the documentation
  2. Set Strip Style to "Non-Global Symbols" as mentioned in
    https://github.com/dart-lang/native/issues/897
  3. Added -force_load flag in podspec

I created a minimal project that can be used to reproduce the error here: https://github.com/jvkolyadich/libndi_bindings

flutter doctor -v output:

[✓] Flutter (Channel stable, 3.0.2, on macOS 12.5 21G72 darwin-x64, locale en-US)
    • Flutter version 3.0.2 at /Users/james/Development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision cd41fdd495 (10 weeks ago), 2022-06-08 09:52:13 -0700
    • Engine revision f15f824b57
    • Dart version 2.17.3
    • DevTools version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/james/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] VS Code (version 1.70.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.46.0

[✓] Connected device (3 available)
    • iPad Air (mobile) • 3aa580ecb4df5acd695435dc8551d32523b3ac07 • ios            • iOS 12.5.5 16H62
    • macOS (desktop)   • macos                                    • darwin-x64     • macOS 12.5 21G72
      darwin-x64
    • Chrome (web)      • chrome                                   • web-javascript • Google Chrome
      104.0.5112.79

[✓] HTTP Host Availability
    • All required HTTP hosts are available

• No issues found!
dcharkes commented 2 years ago

My most recent setup for static libraries in iOS/MacOS is:

s.pod_target_xcconfig = {
--
  | 'DEFINES_MODULE' => 'YES',
  | 'EXCLUDED_ARCHS[sdk=iphonesimulator*]' => 'i386',
  | "OTHER_LDFLAGS[sdk=iphoneos*]" => "-force_load $(PODS_TARGET_SRCROOT)/Frameworks/mylib_staticlib.xcframework/ios-arm64_armv7/libmylib_staticlib.a",
  | "OTHER_LDFLAGS[sdk=iphonesimulator*]" => "-force_load $(PODS_TARGET_SRCROOT)/Frameworks/mylib_staticlib.xcframework/ios-arm64_x86_64-simulator/libmylib_staticlib.a",
  | }
  | s.vendored_frameworks = 'Frameworks/mylib_staticlib.xcframework'

The framework is created by running the following steps from a C source and CMake build.

/Applications/CMake.app/Contents/bin/cmake -S /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/src/mylib_staticlib -B /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/ios_arm/ -GXcode -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_DEPLOYMENT_TARGET=10.0 -DCMAKE_INSTALL_PREFIX=`pwd`/_install -DCMAKE_XCODE_ATTRIBUTE_ONLY_ACTIVE_ARCH=NO -DCMAKE_IOS_INSTALL_COMBINED=YES -DCMAKE_OSX_ARCHITECTURES=armv7 -DCODE_SIGN_IDENTITY=<...>
/Applications/CMake.app/Contents/bin/cmake -S /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/src/mylib_staticlib -B /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/ios_arm64/ -GXcode -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_DEPLOYMENT_TARGET=10.0 -DCMAKE_INSTALL_PREFIX=`pwd`/_install -DCMAKE_XCODE_ATTRIBUTE_ONLY_ACTIVE_ARCH=NO -DCMAKE_IOS_INSTALL_COMBINED=YES -DCMAKE_OSX_ARCHITECTURES=arm64 -DCODE_SIGN_IDENTITY=<...>
/Applications/CMake.app/Contents/bin/cmake -S /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/src/mylib_staticlib -B /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iossimulator_arm64/ -GXcode -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_DEPLOYMENT_TARGET=10.0 -DCMAKE_INSTALL_PREFIX=`pwd`/_install -DCMAKE_XCODE_ATTRIBUTE_ONLY_ACTIVE_ARCH=NO -DCMAKE_IOS_INSTALL_COMBINED=YES -DCMAKE_OSX_ARCHITECTURES=arm64
/Applications/CMake.app/Contents/bin/cmake -S /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/src/mylib_staticlib -B /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iossimulator_x64/ -GXcode -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_DEPLOYMENT_TARGET=10.0 -DCMAKE_INSTALL_PREFIX=`pwd`/_install -DCMAKE_XCODE_ATTRIBUTE_ONLY_ACTIVE_ARCH=NO -DCMAKE_IOS_INSTALL_COMBINED=YES -DCMAKE_OSX_ARCHITECTURES=x86_64
/Applications/CMake.app/Contents/bin/cmake --build /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iossimulator_arm64/ --target mylib_staticlib -- -sdk iphonesimulator
/Applications/CMake.app/Contents/bin/cmake --build /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/ios_arm/ --target mylib_staticlib -- -sdk iphoneos
/Applications/CMake.app/Contents/bin/cmake --build /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/ios_arm64/ --target mylib_staticlib -- -sdk iphoneos
/Applications/CMake.app/Contents/bin/cmake --build /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iossimulator_x64/ --target mylib_staticlib -- -sdk iphonesimulator
lipo -create /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/ios_arm/Debug-iphoneos/libmylib_staticlib.a /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/ios_arm64/Debug-iphoneos/libmylib_staticlib.a -output /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iosiphoneos/libmylib_staticlib.a
lipo -create /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iossimulator_arm64/Debug-iphonesimulator/libmylib_staticlib.a /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iossimulator_x64/Debug-iphonesimulator/libmylib_staticlib.a -output /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iosiphonesimulator/libmylib_staticlib.a
xcodebuild -create-xcframework -library /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iosiphoneos/libmylib_staticlib.a -library /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/out/iosiphonesimulator/libmylib_staticlib.a -output /Users/dacoharkes/ffi-samples/dacoharkes/native_lib_distribution/mylib_staticlib/ios/Frameworks/mylib_staticlib.xcframework
cmake_minimum_required(VERSION 3.10 FATAL_ERROR)

set(CMAKE_OSX_DEPLOYMENT_TARGET "10.11" CACHE STRING "Minimum OS X deployment version")

project(mylib_staticlib_library VERSION 0.0.1 LANGUAGES C)

add_library(mylib_staticlib STATIC mylib_staticlib.c)

set_target_properties(mylib_staticlib PROPERTIES
    PUBLIC_HEADER mylib_staticlib.h
    OUTPUT_NAME "mylib_staticlib"
    XCODE_ATTRIBUTE_CODE_SIGN_IDENTITY "${CODE_SIGN_IDENTITY}"
)

As far as I know, using a framework is the only way to support both the arm64 device and arm64 simulator for iOS. You can also make this to work without a framework:

  s.vendored_libraries = 'Frameworks/libmylib_staticlib.a'
  s.pod_target_xcconfig = { "OTHER_LDFLAGS" => "-force_load $(PODS_TARGET_SRCROOT)/Frameworks/mylib_staticlib.xcframework/macos-arm64_x86_64/libmylib_staticlib.a" }

Set Strip Style to "Non-Global Symbols"

I believe this is not required.

I created a minimal project that can be used to reproduce the error here: https://github.com/jvkolyadich/libndi_bindings

By downloading the .a in binary, it is harder to troubleshoot what is going on.

Try using the file command to see what architectures are in there:

$ file /Users/dacoharkes/src/dacoharkes/native_lib_distribution/mylib_staticlib/ios/Frameworks/mylib_staticlib.xcframework/ios-arm64_x86_64-simulator/libmylib_staticlib.a
/Users/dacoharkes/src/dacoharkes/native_lib_distribution/mylib_staticlib/ios/Frameworks/mylib_staticlib.xcframework/ios-arm64_x86_64-simulator/libmylib_staticlib.a: Mach-O universal binary with 2 architectures: [x86_64:current ar archive random librarycurrent ar archive random library] [arm64:current ar archive random librarycurrent ar archive random library]
/Users/dacoharkes/src/dacoharkes/native_lib_distribution/mylib_staticlib/ios/Frameworks/mylib_staticlib.xcframework/ios-arm64_x86_64-simulator/libmylib_staticlib.a (for architecture x86_64):  current ar archive random library
/Users/dacoharkes/src/dacoharkes/native_lib_distribution/mylib_staticlib/ios/Frameworks/mylib_staticlib.xcframework/ios-arm64_x86_64-simulator/libmylib_staticlib.a (for architecture arm64):   current ar archive random library

And nm -fC <file> to see what symbols are in there

$ nm -gC /Users/dacoharkes/src/dacoharkes/native_lib_distribution/mylib_staticlib/ios/Frameworks/mylib_staticlib.xcframework/ios-arm64_x86_64-simulator/libmylib_staticlib.a

/Users/dacoharkes/src/dacoharkes/native_lib_distribution/mylib_staticlib/ios/Frameworks/mylib_staticlib.xcframework/ios-arm64_x86_64-simulator/libmylib_staticlib.a(mylib_staticlib.o):
0000000000000000 T _sum_staticlib
jvkolyadich commented 2 years ago

The library I'm using is closed-source, only the header files and the fat static library are provided.

As far as I know, using a framework is the only way to support both the arm64 device and arm64 simulator for iOS. You can also make this to work without a framework:

s.vendored_libraries = 'Frameworks/libmylib_staticlib.a'
s.pod_target_xcconfig = { "OTHER_LDFLAGS" => "-force_load $(PODS_TARGET_SRCROOT)/Frameworks/mylib_staticlib.xcframework/macos-arm64_x86_64/libmylib_staticlib.a" }

This is the way I have the project set up, but I'm still getting the Failed to lookup symbol error.

Running the file command shows that the library has all the needed architectures for ios (armv7, i386, x86_64, arm64)

nm -gC shows that the library contains all the needed symbols, including the one shown in the error message (NDIlib_version)

dcharkes commented 2 years ago

arm64

It will only contain either the one for arm64 iosdevice or arm64 iossimulator. Are you running on the iPad Air from your device list?

The next step would be to see if a self-built static library with just a single symbol works, to see if the issue is with how the static library is bundled with your FFI plugin or if the issue is with the closed source static library.

jvkolyadich commented 2 years ago

It will only contain either the one for arm64 iosdevice or arm64 iossimulator. Are you running on the iPad Air from your device list?

I've tried it on an iPad Air and on a simulator, but I ended up with the same error on both.

The static library might actually contain the arm64 archive for both devices and simulators.

Earlier, I ran into an issue that said it was built for iOS + iOS Simulator. This should not be done according to Apple, but there is a workaround that allows it to be used anyway.

I don't think that this is the cause of the symbol not found issue because it wasn't solved by using a self-built static library.

The next step would be to see if a self-built static library with just a single symbol works...

I built a test library with only the NDIlib_version symbol using the steps from your first comment and used it instead of the closed-source library, but still ended up with the same issue.

I made a new minimal project to test this apart from the specific library I'm trying to use. I started from a new dart_ffi plugin and tried to keep the changes to a minimum. Again, I ran into the same problem.

https://github.com/jvkolyadich/ffi_ios_static

jvkolyadich commented 2 years ago

I was able to get the self-built static library to work by adding its headers to Runner-Bridging-Header.h and using one of the functions from the library inside of AppDelegate.swift. However, doing the same for the closed-source library didn't work.

I'm closing this issue since this is likely a problem with the closed-source library.