Random freezes when opening database file

[jtoppine]

Okay this is pretty hard to get a reproduction as this happens intermittently / randomly.

But it seems the following code sometimes never returns control to the main thread. Throws no errors, just silently fails and freezes the runtime. No high CPU usage or anything, just nothing happens. Even setTimeout's don't trigger when it's stuck (tried to create a 1 second timeout to detect a failure in opening the db, but alas, didn't work as control never returns to event loop if it gets stuck here).

I suspect it has to do with some file locking issues. I tend to have a bunch of deno processes and subprocesses running, and sometimes they may become zombies, and perhaps keep the db file hostage. Because sometimes running sudo killall deno fixes it. Rebooting also fixes it. It might be also related to files not properly being closed/unlocked during deno run --watch . Not sure, just guessing here and it's pretty hard to reproduce reliably.

Most often I run into this issue when coding while running unit tests with `--watch´, so the runtime is restarted again and again pretty often (until the bug is triggered). As a workaround, I try to switch to an in-memory database while developing (set dbfile to undefined). The failure never happens with in-memory databases.

No matter the reason, it would be wonderfull if failure to open the db threw an error instead of waiting indefinitely.

Using 3.7.0 on Deno 1.32.3 on Linux Mint 21.1.

    import * as sqlite from "https://deno.land/x/sqlite@v3.7.0/mod.ts";

    const dbfile = "foo.db"; 
    const db = new sqlite.DB(dbfile);
    db.execute(`
      CREATE TABLE IF NOT EXISTS cache (
        id INTEGER PRIMARY KEY AUTOINCREMENT,
        name TEXT NOT NULL,
        hash TEXT NOT NULL,
        key TEXT NOT NULL,
        options TEXT NOT NULL,
        created INTEGER NOT NULL,
        finished INTEGER NOT NULL,
        ms INTEGER NOT NULL,

        ttl INTEGER NOT NULL,
        usecount INTEGER NOT NULL,
        size INTEGER NOT NULL,
        lastused INTEGER NOT NULL,
        expires INTEGER NOT NULL,

        weight INTEGER NOT NULL,
        remove INTEGER NOT NULL DEFAULT 0
      )
    `);
     console.log("this is never seen if sqlite fails to open the db file... nor an error is thrown, nor control is returned to event loop");

[dyedgreen]

I feel like you’re somehow triggering a hanging flock call? This sounds similar to #174

Are you able to produce a simple reproduction example? Otherwise you can try running with a local version of this and eg adding some logging for the file locking calls to see what’s going on (ideally log the call + your pid so we can check if you have processes that stick around unexpectedly)

[dyedgreen]

The relevant code you’ll want to modify is here: https://github.com/dyedgreen/deno-sqlite/blob/43ca0e04d23195da362d978b0de24880a57f94f1/build/vfs.js#L74

[jtoppine]

Are you able to produce a simple reproduction example?

I spent a good while trying to create a simple reproduction, by spawning a bunch of processes that all create DB instances to the same file. Unfortunately no success so far. A long shot, but I might try involving Deno.test in reproduction, as I feel like the test sanitizers or something might interfere with resource/lock cleanup, and this issue seems to surface (mostly?) during testing.

Thank you for the pointers towards possible culprit code. Might try and take a look if I some beautifull day I manage to create reliable repro

[dyedgreen]

Sounds good! You can also try and run with a debug build during your regular work; it will be pretty spammy but you might get some useful information if / when you run into the problem again!