Failed to load dynamic library (dlopen failed: library libsqlite3.so not found)

alexobviously commented 4 years ago

I've got a flutter app using both sqflite_ffi, with a script that works fine when I run it as pure dart but crashes with this error when I trigger it as part of the flutter app:

I/flutter (18313): [SqfliteFfiException(error, Invalid argument(s): Failed to load dynamic library (dlopen failed: library "/data/data/com.twoplayerchess/lib/libsqlite3.so" not found)} DatabaseException(Invalid argument(s): Failed to load dynamic library (dlopen failed: library "/data/data/com.twoplayerchess/lib/libsqlite3.so" not found)) {}, #0      SqfliteIsolate.handle (package:sqflite_common_ffi/src/isolate.dart:39:9)
I/flutter (18313): <asynchronous suspension>
I/flutter (18313): #1      FfiMethodCallHandler._isolateHandle (package:sqflite_common_ffi/src/database_factory_ffi.dart:54:27)
I/flutter (18313): #2      FfiMethodCallHandler.handleInIsolate (package:sqflite_common_ffi/src/database_factory_ffi.dart:33:26)
I/flutter (18313): #3      databaseFactoryFfiImpl.<anonymous closure> (package:sqflite_common_ffi/src/database_factory_ffi.dart:17:25)
I/flutter (18313): #4      _SqfliteDatabaseFactoryImpl.invokeMethod (package:sqflite_common/src/mixin/factory.dart:20:27)
I/flutter (18313): #5      SqfliteDatabaseMixin.invokeMethod (package:sqflite_common/src/database_mixin.dart:287:15)
I/flutter (18313): #6      SqfliteDatabaseMixin.saf

Flutter doctor -v

[✓] Flutter (Channel stable, 1.20.2, on Mac OS X 10.14.6 18G2022, locale en-GB)
    • Flutter version 1.20.2 at /Users/alex/Developer/flutter
    • Framework revision bbfbf1770c (4 weeks ago), 2020-08-13 08:33:09 -0700
    • Engine revision 9d5b21729f
    • Dart version 2.9.1

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    • Android SDK at /Users/alex/Library/Android/sdk
    • Platform android-29, build-tools 29.0.3
    • Java binary at: /Applications/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.3.1, Build version 11C504
    • CocoaPods version 1.9.1

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 48.0.2
    • Dart plugin version 193.7361
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6222593)

[!] IntelliJ IDEA Community Edition (version 2020.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.dev/intellij-setup/#installing-the-plugins

[✓] VS Code (version 1.48.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.14.1

[✓] Connected device (1 available)
    • Android SDK built for x86 (mobile) • emulator-5554 • android-x86 • Android 10
      (API 29) (emulator)

! Doctor found issues in 1 category.

I'm also using the standard sqflite plugin as part of this app for something totally separate. Don't know if that affects anything but that part is running fine. I'm only using sqflite_ffi for this part because it needs to run outside flutter sometimes too and sqflite doesn't support that. I'm not doing anything special when it crashes, just opening a static asset database.

Aracoix commented 4 years ago

same question

alextekartik commented 4 years ago

This part is handled by sqlite3 package. In the README there is a snipped that shows how to bundle libsqlite3.so with your app that you could try.

See https://github.com/simolus3/sqlite3.dart/issues/8

alextekartik commented 4 years ago

Also I'm not sure on which platform you are getting the issue (linux? android?). I recommend using sqflite (not sqflite_common_ffi) on Android

alexobviously commented 4 years ago

Also I'm not sure on which platform you are getting the issue (linux? android?). I recommend using sqflite (not sqflite_common_ffi) on Android

I'm building for android/ios but I also have some tools that needs to run in dart alone, which is why I wasn't just using sqflite (I have a large static db that is only read from the app but I have a dart script to build it).

I have built two separate versions of the sqlite interface now, one using sqflite, but I've run into a different problem - it doesn't seem like it works in an isolate, and that's unfortunately necessary for my application. I've had a bit of a look around here and correct me if I'm wrong but it seems like sqlite doesn't work outside of the main isolate at all?

So I think I need to find a different type of database for this app. I'm trying Hive at the moment but it seems like it suffers from the same problem, so if anyone has any suggestions I'd appreciate them.

alextekartik commented 4 years ago

correct me if I'm wrong but it seems like sqlite doesn't work outside of the main isolate at all?

That's correct. sqflite has its own native threading mechanism and should be used from the main isolate only. (sqflite_common_ffi runs in a separate isolate but must also be used from the main isolate only)

bksubhuti commented 3 years ago

I have a multiplatform app that I'm writing and doing a proof of concept. I have an app that has a prebuilt db, yet most of the samples, simply build the db in code. In android and ios, you have to copy the prebuilt db from the assets folder to the path that the library gives you . The copying is done byte by byte. After that copy is done, one can do the db stuff as expected and as the tutorials show.

So the question is how to do this? Is there an example for a

Using an already populated db that has never been used before
android/ios (mobile)
same code for desktop.. win/linux.

My sample code for proof of concept works with the android OS. I want it to work with the ffi on linux but there is no code example. my git repo for such and example is located at: https://github.com/bksubhuti/mydbtest

alextekartik commented 3 years ago

@bksubhuti I don't have an example for your exact situation but you can find:

a generic solution for opening a database from assets
a simple app that works on all platforms (desktop/mobile, even linux). You just have to re-create the project.

bksubhuti commented 3 years ago

My repo I posted above https://github.com/bksubhuti/mydbtest does copy from the assets using the same technique as the first link you posted. Everyone uses the same example which is mostly the same as this example. However that is only for Android & iOS. . I will look at the second link you gave .but I suspect it is done without a pre-populated assets db as the first example used. Perhaps.one does not need to copy a db to the runtime area like mobiles do.

I'm totally new to flutter and just doing a test for proof of concept before we rewrite a Buddhist Pali reading app (tipitaka Pali Projector). So far I'm impressed and pro-flutter. I'll come.back to report later tonight. Using the ffi still on the mobile emulator is different and there an error as reported on this thread.

alextekartik commented 3 years ago

I will look at the second link you gave .but I suspect it is done without a pre-populated assets db

Correct if just show how to use sqflite on linux

Perhaps.one does not need to copy a db to the runtime area like mobiles do.

Correct, the code (to copy the asset database to a writable location) should be similar, you need to copy the asset file somewhere. I guess you could use path_provider to find the proper destination on Desktop.

bksubhuti commented 3 years ago

Well then if I was I was correct as suspected (not from assets). then can you make a small example that opens a single table query from prepopulated database from assets and works on all platforms? I suspect this has never been done before and no example code exist nor has this been tested. It would be required for the ffi library to say "functional" on all platforms. Since I'm one or two weeks into flutter and a few days into SQLite, this would be a difficult task for me.

alextekartik commented 3 years ago

I follow the doc here on how to use sqflite on linux and was able to run your example on linux with very few changes. I created a pull request so you can see the changes made.

can you make a small example that opens a single table query from prepopulated database from assets and works on all platforms?

Hard for me to create an example for all cases unfortunately. I develop and maintain this library in my free time. However examples are welcome!

It would be required for the ffi library to say "functional" on all platforms.

I know the documentation is poor. The notepad example is a way to say "functional" on all platforms. Copying an asset file is not sqflite specific. So maybe you can find more information in flutter docs regarding how to deal with assets: https://flutter.dev/docs/development/ui/assets-and-images

bksubhuti commented 3 years ago

That is excellent. I'm new to GitHub and need to figure out how to do admin for pull req . Not sure what that is It is easier to make you a contributer to the main repo since this is just test stuff anyways. . I'm also part time doing sw dev and a monk full time. I have been away from programming since the 90's. Flutter and even SQLite are well documented compared to other FOSS platforms. But my old days were msvc++ in the 90's . I'm learning now as I try to make free religious reading apps for my own use and others too.

bksubhuti commented 3 years ago

I pulled the new project, but the flutter run -d linux failed. I did this with another project.. got a similar error in the terminal command window, however, the project worked and ran. I did a flutter upgrade but it still gave the error and launched the mydb app but did not do a query. However, your changes did work with the android emulator

There was a note on stackexhange about using AUR instead of snap for flutter install. I hope not. I did the upgrade, not sure what more to do.

$ flutter run -d linux Launching lib/main.dart on Linux in debug mode... /snap/flutter/current/usr/bin/ld: warning: /usr/lib/gcc/x86_64-linux-gnu/10/../../../x86_64-linux-gnu/crt1.o: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0008002 /snap/flutter/current/usr/bin/ld: warning: //lib/x86_64-linux-gnu/libdl.so.2: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0008002 Building Linux application...
Syncing files to device Linux...

alextekartik commented 3 years ago

We are going away from the original issue here sorry. You should first try to build a linux project without sqflite. If necessary you might have to delete the linux folder and re-create the project. If you manage to run and build a linux project without sqflite and fail when adding sqflite, please open a new issue.

bksubhuti commented 3 years ago

I said in the previous that i did build an app without sqlite and got the same error, but it did run. When i did linux build, it ran but hung on the listbuild from the db. I changed the channel from master to stable (and then did flutter upgrade) but it still failed in the same way. However, I trust that you tested in linux, because a couple of generated files popped into the linux folder when I did the pull. Linux is still new as well, so maybe that is the problem. Did you test on Linux? If so, i think , yes this issue is closed.

bksubhuti commented 3 years ago

I was able to get the project to work on ubuntu 21.04 by removing the flutter snap and installing the flutter gz downloaded from the flutter website. Then I changed the path to reflect the new flutter path and rebooted

I was able run a base app with zero errors showing

However, the mydbtest was still hanging so I ran the debugger and stepped into the code. It could not find libsqlite3.so shared library and threw and exception when trying to open or get the path.. i forget. However, I obviously had sqlite3 tools installed and could open the db and run queries through the command line. Reinstalling sqlite3 was not working and dangerous for a purge.

I installed libsqlite3-dev and ran sudo apt-cache policy sqlite3 and everything worked.. So maybe this needs documentation or working with the ubuntu people.

I appreciate you making the mydbtest program working on your end (and now my end too). It is amazing that flutter has so much support. I'm quite sold on it, but still testing.

by installing the following:

$ sudo apt-get install sqlite3 libsqlite3-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
sqlite3 is already the newest version (3.34.1-3).
The following packages were automatically installed and are no longer required:
  html2text libgranite-common libgranite5 linux-headers-5.11.0-18
  linux-headers-5.11.0-18-generic linux-image-5.11.0-18-generic
  linux-modules-5.11.0-18-generic linux-modules-extra-5.11.0-18-generic
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
  libsqlite3-dev
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 753 kB of archives.
After this operation, 2,622 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://us.archive.ubuntu.com/ubuntu hirsute/main amd64 libsqlite3-dev amd64 3.34.1-3 [753 kB]
Fetched 753 kB in 5s (142 kB/s)          
Selecting previously unselected package libsqlite3-dev:amd64.
(Reading database ... 346361 files and directories currently installed.)
Preparing to unpack .../libsqlite3-dev_3.34.1-3_amd64.deb ...
Unpacking libsqlite3-dev:amd64 (3.34.1-3) ...
Setting up libsqlite3-dev:amd64 (3.34.1-3) ...

$  sudo apt-cache policy sqlite3
sqlite3:
  Installed: 3.34.1-3
  Candidate: 3.34.1-3
  Version table:
 *** 3.34.1-3 500
        500 http://us.archive.ubuntu.com/ubuntu hirsute/main amd64 Packages
        100 /var/lib/dpkg/status

alextekartik commented 3 years ago

@bksubhuti

I installed libsqlite3-dev and ran sudo apt-cache policy sqlite3 and everything worked.. So maybe this needs documentation or working with the ubuntu people.

I was not aware of the need of calling: sudo apt-cache policy sqlite3. Sorry for the poor documentation. Some notes on Ubuntu support are here: https://github.com/tekartik/sqflite/blob/master/sqflite_common_ffi/README.md#linux

PR are welcome to improve the documentation!

bksubhuti commented 3 years ago

I'm not sure if apt-cache policy sqlite3 is required as I installed the dev file at the same time.

However, the sqlite3-dev file is probably needed for sure and I didn't know that.

I do have it working now (on my machine) which is most important

I will try to change the doc and imply for the next person to test if the apt-cache policy sqlite3 is needed.

I don't know pull-requests but i can learn . Surely it is easier than flutter SQLite.

ante Bhikkhu Subhūti

Website: https://AmericanMonk.org

On Thu, Aug 5, 2021, 14:16 Alexandre Roux @.***> wrote:

@bksubhuti https://github.com/bksubhuti

I installed libsqlite3-dev and ran sudo apt-cache policy sqlite3 and everything worked.. So maybe this needs documentation or working with the ubuntu people.

I was not aware of the need of calling: sudo apt-cache policy sqlite3. Sorry for the poor documentation. Some notes on Ubuntu support are here: https://github.com/tekartik/sqflite/blob/master/sqflite_common_ffi/README.md#linux

PR are welcome to improve the documentation!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tekartik/sqflite/issues/505#issuecomment-893242189, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEKXQWBSKHI3GRDVYWPTFGLT3I6WDANCNFSM4RFF3MQQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

tekartik / sqflite

Failed to load dynamic library (dlopen failed: library libsqlite3.so not found) #505