search

simolus3 / sqlite3.dart

sqlite3 bindings for Dart
MIT License
222 stars 74 forks source link

"Failed to load dynamic library" error on Android API 35 emulator with 16KiB page size #257

Open ko16a46 opened 3 days ago

ko16a46 commented 3 days ago

Hello, we've been doing a round of updating our apps' Flutter version, deps, etc. and we've been getting errors on our app with drift, using sqlite3 under the hood.

Notably the error only appears on the Android 15 (API 35) emulator so far, not on our real devices with older Android versions (I tested on my Galaxy S24+ with Android 14), and not on my Android API 32 emulator.

The error is as follows:

E/flutter ( 9614): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Failed to load dynamic library '/data/data/com.kohlerpower.dealerservice/lib/libsqlite3.so': dlopen failed: library "/data/data/com.kohlerpower.dealerservice/lib/libsqlite3.so" not found
E/flutter ( 9614): #0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:11:43)
E/flutter ( 9614): #1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:22:12)
E/flutter ( 9614): #2      _defaultOpen (package:sqlite3/src/ffi/load_library.dart:40:29)
E/flutter ( 9614): #3      OpenDynamicLibrary.openSqlite (package:sqlite3/src/ffi/load_library.dart:127:12)
E/flutter ( 9614): #4      sqlite3 (package:sqlite3/src/ffi/api.dart:13:39)
E/flutter ( 9614): #5      _NativeDelegate.openDatabase (package:drift/native.dart:358:12)
E/flutter ( 9614): #6      Sqlite3Delegate.open (package:drift/src/sqlite3/database.dart:79:19)
E/flutter ( 9614): #7      DelegatedDatabase.ensureOpen.<anonymous closure> (package:drift/src/runtime/executor/helpers/engines.dart:442:22)
E/flutter ( 9614): <asynchronous suspension>
E/flutter ( 9614): #8      _AsyncCompleter.complete (dart:async/future_impl.dart:41:3)
E/flutter ( 9614): <asynchronous suspension>

We create our database in a fairly standard way (using drift):

/// Provides a real connection that uses a shared SQL database
var realConnectionProvider = Provider<QueryExecutor>((ref) {
  open.overrideFor(OperatingSystem.windows, _openOnWindows);

  return LazyDatabase(() async {
    final dbFolder = await getApplicationDocumentsDirectory();
    final file = File(p.join(dbFolder.path, _kDatabaseName));
    return NativeDatabase(file);
  });
});

Here are some of the relevant deps in our pubspec.yaml:

  drift: ^2.13.0
  sqlite3: ^2.1.0
  sqlite3_flutter_libs: ^0.5.0

And pubspec.lock (which should be up to date as we just ran flutter pub upgrade --major-versions):

  drift:
    dependency: transitive
    description:
      name: drift
      sha256: d6ff1ec6a0f3fa097dda6b776cf601f1f3d88b53b287288e09c1306f394fb1b3
      url: "https://pub.dev"
    source: hosted
    version: "2.20.3"
  sqflite:
    dependency: transitive
    description:
      name: sqflite
      sha256: "79a297dc3cc137e758c6a4baf83342b039e5a6d2436fcdf3f96a00adaaf2ad62"
      url: "https://pub.dev"
    source: hosted
    version: "2.4.0"
  sqflite_android:
    dependency: transitive
    description:
      name: sqflite_android
      sha256: "78f489aab276260cdd26676d2169446c7ecd3484bbd5fead4ca14f3ed4dd9ee3"
      url: "https://pub.dev"
    source: hosted
    version: "2.4.0"
  sqflite_common:
    dependency: transitive
    description:
      name: sqflite_common
      sha256: "4468b24876d673418a7b7147e5a08a715b4998a7ae69227acafaab762e0e5490"
      url: "https://pub.dev"
    source: hosted
    version: "2.5.4+5"
  sqflite_darwin:
    dependency: transitive
    description:
      name: sqflite_darwin
      sha256: "769733dddf94622d5541c73e4ddc6aa7b252d865285914b6fcd54a63c4b4f027"
      url: "https://pub.dev"
    source: hosted
    version: "2.4.1-1"
  sqflite_platform_interface:
    dependency: transitive
    description:
      name: sqflite_platform_interface
      sha256: "8dd4515c7bdcae0a785b0062859336de775e8c65db81ae33dd5445f35be61920"
      url: "https://pub.dev"
    source: hosted
    version: "2.4.0"
  sqlite3:
    dependency: transitive
    description:
      name: sqlite3
      sha256: "45f168ae2213201b54e09429ed0c593dc2c88c924a1488d6f9c523a255d567cb"
      url: "https://pub.dev"
    source: hosted
    version: "2.4.6"
  sqlite3_flutter_libs:
    dependency: transitive
    description:
      name: sqlite3_flutter_libs
      sha256: "62bbb4073edbcdf53f40c80775f33eea01d301b7b81417e5b3fb7395416258c1"
      url: "https://pub.dev"
    source: hosted
    version: "0.5.24"

Flutter doctor output (we are now on the latest version of Flutter):

KO16A46@LXHP0G54C6N dealer_service % flutter doctor -v
[✓] Flutter (Channel stable, 3.24.3, on macOS 15.0.1 24A348 darwin-arm64, locale en-US)
    • Flutter version 3.24.3 on channel stable at /Users/KO16A46/Development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 2663184aa7 (5 weeks ago), 2024-09-11 16:27:48 -0500
    • Engine revision 36335019a8
    • Dart version 3.5.3
    • DevTools version 2.37.3

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/KO16a46/Library/Android/sdk
    • Platform android-35, build-tools 34.0.0
    • ANDROID_HOME = /Users/KO16a46/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
    • Xcode at /Applications/Xcode-16.0.0.app/Contents/Developer
    • Build 16A242d
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2024.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)

[✓] IntelliJ IDEA Community Edition (version 2022.3.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.94.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.98.0

[✓] Connected device (7 available)
    • SM S926U1 (mobile)              • R5CWC2S9KYF                          • android-arm64  • Android 14 (API 34)
    • sdk gphone16k arm64 (mobile)    • emulator-5554                        • android-arm64  • Android 15 (API 35) (emulator)
    • Brian’s iPhone (mobile)         • 00008110-001824D90C33801E            • ios            • iOS 18.0.1 22A3370
    • iPhone 15 Pro Max (mobile)      • 7E5F4EB4-12C7-4F0E-A56F-250B5682FF40 • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-17-0 (simulator)
    • macOS (desktop)                 • macos                                • darwin-arm64   • macOS 15.0.1 24A348 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad                • darwin         • macOS 15.0.1 24A348 darwin-arm64
    • Chrome (web)                    • chrome                               • web-javascript • Google Chrome 129.0.6668.101

[✓] Network resources
    • All expected network resources are available.

• No issues found!

I tried unzipping the APK that gets generated when running on the API 35 emulator, and I do see libsqlite3.so under each of the lib/<platform>/ folders

Screenshot 2024-10-14 at 4 01 08 PM

Let me know if you need more information. Unfortunately we don't have any real devices with Android 15.0 to test with yet, just emulators so far.

simolus3 commented 2 days ago

Thanks for the report! Are you using an emulator with 16KiB page sizes? (if not sure, run adb shell getconf PAGE_SIZE)? I can't reproduce this with a 4KiB emulator on API 35. And even a fresh flutter create project crashes with 16KiB pages, so I can't diagnose that further.

Are you using drift_flutter or, if not, are you calling applyWorkaroundToOpenSqlite3OnOldAndroidVersions() somewhere?

ko16a46 commented 2 days ago

Hello, yes it turns out I'm using the 16KiB page size on my emulator

KO16A46@LXHP0G54C6N kohler_flutter_apps % adb shell getconf PAGE_SIZE
16384

Hmm interesting that I can get my app to run on the emulator when it sounds like you may be having issues. I tried running a vanilla flutter create app on this 16KiB emulator and it works for me. Note that I am on Flutter 3.24.3.

I did try running my app on an Android API 35 emulator without 16KiB paging and I can confirm that I see no errors. So it sounds like the 16KiB page size is actually the root issue.

Also to answer your other questions, we are actually not using drift_flutter, would you recommend we try that? We are also not calling applyWorkaroundToOpenSqlite3OnOldAndroidVersions() either (although actually we should probably investigate and try using this)

simolus3 commented 2 days ago

Interesting, I have the same Flutter version. Is this an x86 or an arm emulator?

I only mentioned drift_flutter because it invokes the workaround automatically, and I suspect it might be related to this. The method essentially invokes System.loadLibrary in Java, which is not exactly the same thing as dlopen/DynamicLibrary.open because it's also able to open the library from the apk directly without an explicit decompression step. The Android 15 changelog mentioned that method specifically continuing to work with 16 KiB pages, so maybe that fixes this (if it does, it obviously still something worth looking at, ideally this should continue to work automatically).

ko16a46 commented 2 days ago

Noted thanks, I will try out drift_flutter later and see if that affects the issue

Interesting, I have the same Flutter version. Is this an x86 or an arm emulator?

I'm using an arm64 image on my Android emulator. Screenshot 2024-10-15 at 2 41 27 PM

Also note that I am running it on an M1 Macbook Pro.

ko16a46 commented 2 hours ago

Hello,

FYI I tried using driftDatabase() within drift_flutter on my Android API 35 emulator with 16KiB page size

Unfortunately it seems that failed too, but the error message does appear to be more useful

D/nativeloader( 7526): Load /data/app/~~e75lMYiyAe5h8a1LPKr-KA==/com.kohlerpower.dealerservice-j6-CJ_8pWOC3SsHnxkMFpg==/lib/arm64/libsqlite3.so using ns clns-7 from class loader (caller=/data/app/~~e75lMYiyAe5h8a1LPKr-KA==/com.kohlerpower.dealerservice-j6-CJ_8pWOC3SsHnxkMFpg==/base.apk!classes2.dex): dlopen failed: empty/missing DT_HASH/DT_GNU_HASH in "/data/app/~~e75lMYiyAe5h8a1LPKr-KA==/com.kohlerpower.dealerservice-j6-CJ_8pWOC3SsHnxkMFpg==/lib/arm64/libsqlite3.so" (new hash type from the future?)
E/flutter ( 7526): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(java.lang.UnsatisfiedLinkError: dlopen failed: empty/missing DT_HASH/DT_GNU_HASH in "/data/app/~~e75lMYiyAe5h8a1LPKr-KA==/com.kohlerpower.dealerservice-j6-CJ_8pWOC3SsHnxkMFpg==/lib/arm64/libsqlite3.so" (new hash type from the future?), null, null, null)
E/flutter ( 7526): #0      StandardMethodCodec.decodeEnvelope (package:flutter/src/services/message_codecs.dart:648:7)
E/flutter ( 7526): #1      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:334:18)
E/flutter ( 7526): <asynchronous suspension>
E/flutter ( 7526): #2      applyWorkaroundToOpenSqlite3OnOldAndroidVersions (package:sqlite3_flutter_libs/sqlite3_flutter_libs.dart:32:5)
E/flutter ( 7526): <asynchronous suspension>
E/flutter ( 7526): #3      driftDatabase.<anonymous closure> (package:drift_flutter/src/native.dart:42:9)
E/flutter ( 7526): <asynchronous suspension>
E/flutter ( 7526): #4      new DatabaseConnection.delayed.<anonymous closure> (package:drift/src/runtime/api/connection.dart:67:39)
E/flutter ( 7526): <asynchronous suspension>
E/flutter ( 7526): 
E/flutter ( 7526): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(java.lang.UnsatisfiedLinkError: dlopen failed: empty/missing DT_HASH/DT_GNU_HASH in "/data/app/~~e75lMYiyAe5h8a1LPKr-KA==/com.kohlerpower.dealerservice-j6-CJ_8pWOC3SsHnxkMFpg==/lib/arm64/libsqlite3.so" (new hash type from the future?), null, null, null)
E/flutter ( 7526): #0      StandardMethodCodec.decodeEnvelope (package:flutter/src/services/message_codecs.dart:648:7)
E/flutter ( 7526): #1      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:334:18)
E/flutter ( 7526): <asynchronous suspension>
E/flutter ( 7526): #2      applyWorkaroundToOpenSqlite3OnOldAndroidVersions (package:sqlite3_flutter_libs/sqlite3_flutter_libs.dart:32:5)
E/flutter ( 7526): <asynchronous suspension>
E/flutter ( 7526): #3      driftDatabase.<anonymous closure> (package:drift_flutter/src/native.dart:42:9)
E/flutter ( 7526): <asynchronous suspension>
E/flutter ( 7526): #4      new DatabaseConnection.delayed.<anonymous closure> (package:drift/src/runtime/api/connection.dart:67:39)
E/flutter ( 7526): <asynchronous suspension>
E/flutter ( 7526):

I searched around and it appears that Realm had a similar issue here with the suggestion to compile libraries using 16KB ELF alignment as a fix (as seen in this Realm PR).

I suppose you'd need to recompile your sqlite-native-libraries with 16KB ELF alignment to fix the issue? At least, that's what it looks like to me, but admittedly it's a bit out of my depth hehe. Feel free to message if you need help with testing