powersync-ja / sqlite_async.dart

High-performance asynchronous interface for SQLite on Dart & Flutter
https://pub.dev/packages/sqlite_async
MIT License
45 stars 9 forks source link

Error in some tests on macOS #61

Open js2702 opened 1 month ago

js2702 commented 1 month ago

We are seeing some errors while running the tests on macOS. We tested it in both Intel and Arm devices and we are seeing the same failing tests.

For instance in the sqlite_async/test/native/basic_test.dart, both the test named "closing" and "lockTimeout" are failing with the following stacktrace:

00:00 +12 -1: Basic Tests lockTimeout [E]                                                                                                                                                                                                                                                                                 
  SqliteException(14): while executing, unable to open database file, unable to open database file (code 14)
    Causing statement: PRAGMA synchronous = NORMAL, parameters: 
  package:sqlite3/src/implementation/exception.dart 75:3                              throwException
  package:sqlite3/src/implementation/database.dart 244:9                              DatabaseImplementation.execute
  package:sqlite_async/src/common/abstract_open_factory.dart 94:14                    AbstractDefaultSqliteOpenFactory.open
  test/utils/native_test_utils.dart 28:22                                             TestSqliteOpenFactory.open
  package:sqlite_async/src/native/database/native_sqlite_connection_impl.dart 267:33  _sqliteConnectionIsolate

May be worth adding some macOS runners to the github actions tests.

js2702 commented 1 month ago

One thing we noticed is that, for example in the "closing" test if we change the following lines

      final future1 = db.get('SELECT test_sleep(10) as sleep');
      final future2 = db.get('SELECT test_sleep(10) as sleep');

to

      final future1 = db.execute('SELECT test_sleep(10) as sleep');
      final future2 = db.execute('SELECT test_sleep(10) as sleep');

It passes the test fine.

rkistner commented 1 month ago

Changing from get to execute changes which connection the query executes on, which fundamentally changes the test (it specifically tests querying on multiple connections concurrently using get).

The issue appears to be just a race condition when opening multiple connections.

In powersync we automatically retry the open function for a while, to work around issues like this: https://github.com/powersync-ja/powersync.dart/blob/2ccf2396b275e85122f549bd35673f35b8b9c2f8/packages/powersync/lib/src/open_factory/abstract_powersync_open_factory.dart#L52-L68

We'll see if we can reproduce, and add the same workaround in the base library, and do the same for code 14.