WiseLibs / better-sqlite3

The fastest and simplest library for SQLite3 in Node.js.
MIT License
5.23k stars 390 forks source link

WAL mode - Random Transaction (Insert Many) do nothing... #1116

Closed Vance-ng-vn closed 7 months ago

Vance-ng-vn commented 7 months ago

I've just started using better-sqlite3, beginning with version 9.x, and there's a pretty annoying bug: 'transaction function does nothing'. And there's no error message at all!

My suspicion: during the transaction, a select statement might be causing this disruption.

And here is my source code:

function set(table, keys = Array, values = Array, where, value) {
    if(!where) {
        const sql = `INSERT OR IGNORE INTO ${table} (${keys.join(',')}) VALUES (${'?,'.repeat(keys.length - 1)}?)`;
        const insert = db.prepare(sql);
        return insert.run(values);
    } else {
        keys = keys.map(key => `${key} = ?`).join(',');
        const sql = `UPDATE ${table} SET ${keys}, updated_at = CURRENT_TIMESTAMP WHERE ${where} = ?`;
        const update = db.prepare(sql);
        return update.run([...values, value]);
    }
}

function InsertMany(table, keys = Array, values = Array) {
    let inserted = 0;
    const insertMany = db.transaction((values) => {
        for(const value of values) {
            let _ins = set(table, keys, value);
            if(_ins.changes) inserted += _ins.changes;;
        }
    });
    insertMany(values);
    return { changes: inserted };
}
const insert = filter.map(sub => [id, sub.lang, sub.title, sub.path]);
let _ins_m = db.InsertMany(db.Tables.Subtitles, ['id', 'lang', 'title', 'path'], insert);
if(!_ins_m.changes) console.error(new Error("Unknow Error, InsertMany failed!"));

image

Is there an older version that works well with WAL Mode?

Docker: alphine 3.17.5 Nodejs: 18.18.2 better-sqlite3: v9.x (now, it's 9.2.2)

Prinzhorn commented 7 months ago

Please provide a minimal self-contained example that demonstrates the problem (a single js file that sets up everything). I'm certain that either your code or your assumptions are broken. For example INSERT OR IGNORE INTO might cause nothing to be inserted at all when a duplicate key is inserted. Assuming InsertMany failed just because changes === 0 is flawed.

Vance-ng-vn commented 7 months ago

Ah, as you say, my assumptions were not accurate. Thank you very much.