`libsqlite3.so` dlopen failed with `sqlcipher_flutter_libs`

[FMorschel]

libsqlite3.so dlopen failed with sqlcipher_flutter_libs

I found the docs on the native db for Android, and it sais:

Opening libsqlite3.so fails on some Android 6.0.1 devices. This can be fixed by setting android.bundle.enableUncompressedNativeLibs=false in your gradle.properties file. Note that this will increase the disk usage of your app. See this issue for details.

But I want to point out that this only mentions apps that use sqlite3_flutter_libs. I need my db to be encrypted, so I'm using sqlcipher_flutter_libs as suggested here

The issue mentioned in the docs (cited above) is #420. I've followed the actual suggestion to add android.bundle.enableUncompressedNativeLibs=false on my app gradle.properties but still not working. Any suggestions on how can I proceed with this?

I'm using:

dependencies:
  #...
  drift: ^2.11.1
  sqlcipher_flutter_libs: ^0.6.0
  sqlite3: any
  #...
dev_dependencies:
  #...
  drift_dev: ^2.11.1
  #...

flutter doctor -v

``` [√] Flutter (Channel stable, 3.13.2, on Microsoft Windows [Version 10.0.22621.2283], locale pt-BR) • Flutter version 3.13.2 on channel stable at C:\src\flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision ff5b5b5fa6 (4 weeks ago), 2023-08-24 08:12:28 -0500 • Engine revision b20183e040 • Dart version 3.1.0 • DevTools version 2.25.0 [√] Windows Version (Installed version of Windows is version 10 or higher) [√] Android toolchain - develop for Android devices (Android SDK version 33.0.0) • Android SDK at C:\Users\felip_0vh5fa6\AppData\Local\Android\sdk • Platform android-33, build-tools 33.0.0 • Java binary at: C:\Users\felip_0vh5fa6\AppData\Local\JetBrains\Toolbox\apps\AndroidStudio\ch-0\222.4459.24.2221.9862592\jbr\bin\java • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-9586694) • All Android licenses accepted. [√] Chrome - develop for the web • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe [√] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.4.5) • Visual Studio at C:\Program Files\Microsoft Visual Studio\2022\Community • Visual Studio Community 2022 version 17.4.33403.182 • Windows 10 SDK version 10.0.19041.0 [√] Android Studio (version 2022.2) • Android Studio at C:\Users\felip_0vh5fa6\AppData\Local\JetBrains\Toolbox\apps\AndroidStudio\ch-0\222.4459.24.2221.9862592 • Flutter plugin version 75.1.1 • Dart plugin version 222.4582 • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-9586694) [√] VS Code (version 1.82.2) • VS Code at C:\Users\felip_0vh5fa6\AppData\Local\Programs\Microsoft VS Code • Flutter extension version 3.72.0 [√] VS Code (version 1.79.0-insider) • VS Code at C:\Users\felip_0vh5fa6\AppData\Local\Programs\Microsoft VS Code Insiders • Flutter extension version 3.64.0 [√] Connected device (4 available) • SM S918B (mobile) • 192.168.200.22:36831 • android-arm64 • Android 13 (API 33) • Windows (desktop) • windows • windows-x64 • Microsoft Windows [Version 10.0.22621.2283] • Chrome (web) • chrome • web-javascript • Google Chrome 116.0.5845.188 • Edge (web) • edge • web-javascript • Microsoft Edge 116.0.1938.81 [√] Network resources • All expected network resources are available. • No issues found! ```

[davidmartos96]

When using sqlcipher_flutter_libs what should be opened is libsqlcipher.so, not libsqlite3.so. Is it possible that you skipped the following part in the docs related to encryption?

You need to call that function at the beginning of your main or at the beginning of your database isolate.

[FMorschel]

I saw this on the Encryption docs, I'm doing this but I have no idea why would it be calling the libsqlite3.so. I'm actually using as well:

  firebase_analytics: ^10.4.5
  firebase_app_check: ^0.1.5+2
  firebase_auth: ^4.7.2
  firebase_core: ^2.15.0
  firebase_dynamic_links: ^5.3.5
  firebase_ui_auth: ^1.7.0
  firebase_ui_localizations: ^1.6.0
  firebase_ui_oauth_google: ^1.2.9
  firebase_ui_oauth: ^1.4.9
  firebase_storage: ^11.2.6

I saw a brief example of the same error while using one of the Firebase packages while I was searching for this problem, I'm not 100% sure which one of the packages because I skipped the example as it was really brief. It had something to do with the Firebase package removing other data from the folder it was using. I made sure that the call you mentioned as well as the db creation call was all called after initializing all Firebase-related packages. I can look for it later in my browser history to update this issue.

[davidmartos96]

Sharing the stack trace you are getting could also help.

In any case, the setup call shouldn't interfere with Firebase as far as I know. Is there any reason you are calling setupSqlCipher after initializing the Firebase packages?

[simolus3]

Yeah it would be great to see the stack trace to see who is trying to open libsqlite3.so.

Are you using drift with a NativeDatabase.inBackground? That one spawns a background isolate which has "fresh" settings on how sqlite3 is loaded. In that case, you could use the isolateSetup parameter to apply the override on the isolate opening the database as well.

[FMorschel]

I'm only setting up sqlcipher after all Firebase initialization because I saw that issue I mentioned before. I revisited it now. It was talking about firebase_admob. I'm not using it specifically, figured another Firebase package could have the same problem, but nothing changed.

Stack trace:

E/flutter (21429): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Failed to load dynamic library '/data/data/io.github.fmorschel.wallet_flow/lib/libsqlite3.so': dlopen failed: library "/data/data/io.github.fmorschel.wallet_flow/lib/libsqlite3.so" not found
E/flutter (21429): #0      DriftCommunication.request (package:drift/src/remote/communication.dart:113:66)
E/flutter (21429): #1      _RemoteQueryExecutor.ensureOpen (package:drift/src/remote/client_impl.dart:158:10)
E/flutter (21429): #2      LazyDatabase.ensureOpen.<anonymous closure> (package:drift/src/utils/lazy_database.dart:61:49)
E/flutter (21429): <asynchronous suspension>
E/flutter (21429): #3      PeopleDao.get (package:wallet_flow/src/db/daos/people_dao.dart:32:20)
E/flutter (21429): <asynchronous suspension>

The start of my main.dart file:

void setupSqlCipher() {
  open.overrideFor(
    OperatingSystem.android,
    () => DynamicLibrary.open('libsqlcipher.so'),
  );
}

Future<void> main() async {
  final googleProvider = GoogleProvider(clientId: googleClientID);
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp(
    options: DefaultFirebaseOptions.currentPlatform,
  );
  await FirebaseAppCheck.instance.activate(
    webRecaptchaSiteKey: 'recaptcha-v3-site-key',
    androidProvider:
        kDebugMode ? AndroidProvider.debug : AndroidProvider.playIntegrity,
    appleProvider: kDebugMode ? AppleProvider.debug : AppleProvider.appAttest,
  );
  FirebaseUIAuth.configureProviders([
    googleProvider,
    PhoneAuthProvider(),
    EmailAuthProvider(),
  ]);
  await FirebaseAnalytics.instance.logAppOpen();
  setupSqlCipher();
  runApp(MainApp(googleProvider: googleProvider));
}

My MainApp (the meaningful things to the DB):

class _MainAppState extends State<MainApp> {
  final walletFlowDatabase = WalletFlowDatabase();

  @override
  Widget build(BuildContext context) {
    return MultiProvider(
      providers: [
        Provider(create: (_) => walletFlowDatabase),
      ],
      builder: (context, _) => SafeArea(
        child: MaterialApp.router(
          routerConfig: router,
          title: 'WalletFlow',
          themeMode: ThemeMode.system,
        ),
      ),
    );
  }

}

My call to PeopleDao.get method:

class _HomePageState extends State<HomePage> {
  late WalletFlowDatabase db;

  @override
  void initState() {
    super.initState();
    db = context.read();
    try {
      db.peopleDao.get(filter: user.uid).then((people) {
        /// go to different pages and such
        log('people: ${jsonEncode([...people])}');
      });
    } catch (e, s) {
      debugPrint('$e\n\n$s');
      rethrow;
    }
  }

}

My database:

@DriftDatabase(
  tables: [
    People,
    Wallets,
    Entries,
  ],
  daos: [
    EntriesDao,
    PeopleDao,
    WalletsDao,
  ],
)
class WalletFlowDatabase extends _$WalletFlowDatabase {
  // We tell the database where to store the data with this constructor.
  WalletFlowDatabase() : super(_openConnection());

  static LazyDatabase _openConnection() {
    // The LazyDatabase util lets us find the right location for the file async.
    return LazyDatabase(() async {
      // Put the database file, called db.sqlite here, into the documents folder
      // for your app.
      final dbFolder = await getApplicationDocumentsDirectory();
      final file = File(p.join(dbFolder.path, 'wallet_flow.db'));
      return NativeDatabase.createInBackground(
        file,
        setup: setup,
      );
    });
  }

  static void setup(Database rawDb) {
    assert(_debugCheckHasCipher(rawDb));
    rawDb.execute("PRAGMA key = '$dbEncryptionPassword';");
  }

  static bool _debugCheckHasCipher(Database database) {
    return database.select('PRAGMA cipher_version;').isNotEmpty;
  }

  @override
  int get schemaVersion => 1;

  @override
  MigrationStrategy get migration {
    return MigrationStrategy(
      beforeOpen: (details) async {
        await customStatement('PRAGMA foreign_keys = ON');
      },
      onCreate: (migrator) async {
        await migrator.createAll();
      },
      onUpgrade: (migrator, from, to) async {},
    );
  }
}

Please let me know if you need any more information and how can I help more.

[FMorschel]

A little more context

Just so that I'm clear, you probably have already guessed it, but I think it's best I say it anyway. I've tested commenting out the

setupSqlCipher();

line on my main function and

setup: setup,

on my _openConnection method and that solved my problem.

Question

Are you using drift with a NativeDatabase.inBackground?

The method you meant to write was createInBackground? Then yes.

Are you suggesting that I remove my

setupSqlCipher();

line on my main function and add a

isolateSetup: setupSqlCipher,

to my _openConnection method?

Comment

I will try this later when I can, but if that's the case, I suggest that you mention this page on the setup part of the encryption so that other users like me don't accidentally do this mistake. I was only "following orders" that I didn't understand 🙃.

[FMorschel]

Yeah it would be great to see the stack trace to see who is trying to open libsqlite3.so.

Are you using drift with a NativeDatabase.inBackground? That one spawns a background isolate which has "fresh" settings on how sqlite3 is loaded. In that case, you could use the isolateSetup parameter to apply the override on the isolate opening the database as well.

It worked! Thank you!

Still want to suggest that this information should be on the Encryprion setup page so that other users like me don't make the same mistake.

I ended up trying things that didn't work at all to solve my problem just because this was something I, as a user, never thought about. My drift database always worked without any issues and I didn't even stop to think that my db creation (the method I looked at only once when starting my app) was making the db on another isolate. This is only due to your remarkable work on this package, maybe with a harder API or less complete docs, I would have found my mistake 😂😂😂.

[simolus3]

Thanks! And yeah, this should be mentioned in the docs, I'll add it.

[hongfeiyang]

I got the same issue currently, it just keep opening the regular libsqlite3.so regardless how I set it up, can anyone help me?

// call this method before using drift
Future<void> setupSqlCipher() async {
  await applyWorkaroundToOpenSqlCipherOnOldAndroidVersions();
  open.overrideFor(OperatingSystem.android, openCipherOnAndroid);
}

bool _debugCheckHasCipher(Database database) {
  // Check that we're actually running with SQLCipher by quering the
  // cipher_version pragma.
  return database.select('PRAGMA cipher_version;').isNotEmpty;
}

/// see: https://github.com/simolus3/drift/blob/develop/examples/encryption/lib/database.dart#L39
QueryExecutor openConnection() {
  return LazyDatabase(() async {
    final dbFolder = await getApplicationDocumentsDirectory();
    // TODO(someone): have a central place to store the db file names when we have more than one dbs
    final file = File(join(dbFolder.path, 'citiesAndTowns.db'));
    if (!file.existsSync()) {
      // TODO(someone): in the future once we decide to ship some preloaded dbs with the app, we dont need to create
      // the db file here. Instead if they are not found, that's a big problem
      await file.create();
    }
    final token = RootIsolateToken.instance;
    return NativeDatabase.createInBackground(
      file,
      isolateSetup: () async {
        BackgroundIsolateBinaryMessenger.ensureInitialized(token!);
        await setupSqlCipher();
        open
          ..overrideFor(OperatingSystem.linux,
              () => DynamicLibrary.open('libsqlcipher.so'))
          ..overrideFor(OperatingSystem.windows,
              () => DynamicLibrary.open('sqlcipher.dll'));
      },
      setup: (db) {
        assert(_debugCheckHasCipher(db),
            'The database was not opened with SQLCipher');

        // Then, apply the key to encrypt the database. Unfortunately, this
        // pragma doesn't seem to support prepared statements so we inline the
        // key.
        // TOOD: very very bad practice! this is for demo purposes only
        const encryptionPassword = 'SECURE PASS';
        final escapedKey = encryptionPassword.replaceAll("'", "''");
        db.execute("pragma key = '$escapedKey'");

        // Test that the key is correct by selecting from a table
        // db.execute('SELECT * FROM Site LIMIT 1');
        final queryResults = db.select('SELECT * FROM au_towns LIMIT 1');
        if (queryResults.isEmpty) {
          throw Exception('Failed to decrypt database');
        }
      },
    );
  });
}

[simolus3]

From a quick look the code looks correct to me, is this failing on Linux or Android? In a real Flutter context or when running as a unit test?