simolus3 / sqlite3.dart

sqlite3 bindings for Dart
MIT License
227 stars 78 forks source link

Select statement causes a "database disk image is malformed" error on web. #105

Open bradleybauer opened 2 years ago

bradleybauer commented 2 years ago

Hi, I've recently switched to using the experimental WasmDatabase implementation for easy cross platform support. Everything works as expected on windows but on chrome (and firefox) I get this error after issuing a select statement.

Uncaught Error: SqliteException(11): database disk image is malformed, database disk image is malformed (code 11)
  Causing statement: SELECT * FROM market_orders_cache;

I use this code to add approximately 160,000 rows to the table

  Future<void> setOrders(Map<int, List<Order>> allOrders) async {
    await _clearCache(cache.marketOrdersCache);
    final inserts = <MarketOrdersCacheCompanion>[];
    allOrders.forEach((_, orders) {
      inserts.addAll(orders.map((o) => MarketOrdersCacheCompanion.insert(
        typeID: o.typeID,
        systemID: o.systemID,
        regionID: o.regionID,
        isBuy: o.isBuy,
        price: o.price,
        volumeRemaining: o.volumeRemaining,
      )));
    });
    await cache.batch((batch) {
      batch.insertAll(cache.marketOrdersCache, inserts);
    });
  }

Later I use a basic select statement which causes the error.

    final rows = await cache.select(cache.marketOrdersCache).get();

Here is my project for reference if needed.

The issue occurs on the latest drift and flutter master channel versions.

bradleybauer commented 2 years ago

I changed the drift database implementation back to WebDatabase with IndexedDb but without webworkers. Now everything works as expected and is relatively easy to use. Thank you for the great library.

simolus3 commented 2 years ago

Sorry for the slow response, I was pretty busy during the last days and didn't have a chance to take a good look at this yet. I definitely want to fix this in the wasm implementation though :D

Can I reproduce this by checking a revision of your project before https://github.com/bradleybauer/evereactor/commit/2d31defa1c03fead36eeef7b0b09503b527cdac8? Anything specific that I need to do to trigger this bug?

bradleybauer commented 2 years ago

No problem at all.

Yeah that would work. One thing you can do to make the build easier is download lib\sde.dart from the master channel (it is not in earlier revisions). This way you can skip the generation of that file. It has not changed recently so does not hurt reproducibility.

Then do the following

flutter pub get
flutter pub run build_runner build --delete-conflicting-outputs
flutter build web
cd build/web
python -m http.server

Open the app at http://localhost:8000/. Click the "Get Market Data" button in the upper right of the app. Let the progress bar finish (downloads a bit of data from the internet). Reload the webpage and you should see the error in the console.

The current working version is at evereactor.com