capacitor-community / sqlite

Community plugin for native & electron SQLite databases
MIT License
433 stars 106 forks source link

Android - net.sqlcipher.database.SQLiteException: unable to close due to unfinalized statements or unfinished backups: sqlite3_close() failed #36

Closed Karyfars closed 3 years ago

Karyfars commented 3 years ago

Describe the bug On "larger" queries (e.g. a run call with 20 Create Table statements or a executeSet call with 30 Update queries) the android version will always throw an exception due to unfinalized statements or unfinished backups.

The exact Exceptions always thrown are:

E/Capacitor: Serious error executing plugin
    java.lang.reflect.InvocationTargetException
        at java.lang.reflect.Method.invoke(Native Method)
        at com.getcapacitor.PluginHandle.invoke(PluginHandle.java:99)
        at com.getcapacitor.Bridge$1.run(Bridge.java:521)
        at android.os.Handler.handleCallback(Handler.java:883)
        at android.os.Handler.dispatchMessage(Handler.java:100)
        at android.os.Looper.loop(Looper.java:214)
        at android.os.HandlerThread.run(HandlerThread.java:67)
     Caused by: net.sqlcipher.database.SQLiteException: unable to close due to unfinalized statements or unfinished backups: sqlite3_close() failed
        at net.sqlcipher.database.SQLiteDatabase.dbclose(Native Method)
        at net.sqlcipher.database.SQLiteDatabase.onAllReferencesReleased(SQLiteDatabase.java:517)
        at net.sqlcipher.database.SQLiteDatabase.close(SQLiteDatabase.java:1405)
        at com.getcapacitor.community.database.sqlite.cdssUtils.SQLiteDatabaseHelper.execSet(SQLiteDatabaseHelper.java:271)
        at com.getcapacitor.community.database.sqlite.CapacitorSQLite.executeSet(CapacitorSQLite.java:173)
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.getcapacitor.PluginHandle.invoke(PluginHandle.java:99) 
        at com.getcapacitor.Bridge$1.run(Bridge.java:521) 
        at android.os.Handler.handleCallback(Handler.java:883) 
        at android.os.Handler.dispatchMessage(Handler.java:100) 
        at android.os.Looper.loop(Looper.java:214) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 

E/AndroidRuntime: FATAL EXCEPTION: CapacitorPlugins
    Process: io.ionic.starter, PID: 28875
    java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at com.getcapacitor.Bridge$1.run(Bridge.java:530)
        at android.os.Handler.handleCallback(Handler.java:883)
        at android.os.Handler.dispatchMessage(Handler.java:100)
        at android.os.Looper.loop(Looper.java:214)
        at android.os.HandlerThread.run(HandlerThread.java:67)
     Caused by: java.lang.reflect.InvocationTargetException
        at java.lang.reflect.Method.invoke(Native Method)
        at com.getcapacitor.PluginHandle.invoke(PluginHandle.java:99)
        at com.getcapacitor.Bridge$1.run(Bridge.java:521)
        at android.os.Handler.handleCallback(Handler.java:883) 
        at android.os.Handler.dispatchMessage(Handler.java:100) 
        at android.os.Looper.loop(Looper.java:214) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
     Caused by: net.sqlcipher.database.SQLiteException: unable to close due to unfinalized statements or unfinished backups: sqlite3_close() failed
        at net.sqlcipher.database.SQLiteDatabase.dbclose(Native Method)
        at net.sqlcipher.database.SQLiteDatabase.onAllReferencesReleased(SQLiteDatabase.java:517)
        at net.sqlcipher.database.SQLiteDatabase.close(SQLiteDatabase.java:1405)
        at com.getcapacitor.community.database.sqlite.cdssUtils.SQLiteDatabaseHelper.execSet(SQLiteDatabaseHelper.java:271)
        at com.getcapacitor.community.database.sqlite.CapacitorSQLite.executeSet(CapacitorSQLite.java:173)
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.getcapacitor.PluginHandle.invoke(PluginHandle.java:99) 
        at com.getcapacitor.Bridge$1.run(Bridge.java:521) 
        at android.os.Handler.handleCallback(Handler.java:883) 
        at android.os.Handler.dispatchMessage(Handler.java:100) 
        at android.os.Looper.loop(Looper.java:214) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 

When i execute the statements in executeSet one after the other with run, everything finishes without errors.

Smartphone:

Any idea why this could happen?

Thanks in advance!

jepiqueau commented 3 years ago

@Karyfars i will have a look and come back

jepiqueau commented 3 years ago

@Karyfars which version of the plugin are you using? make sure that you run the following after you install npm i --save @capacitor-community/sqlite@latest

npx cap update
npm run build
npx cap copy
npx cap open android

i test this and it works fine 22 tables and 55 (insert/update) in executeSet. the tables are created and the corresponding data are loaded

  async testExecuteSet(): Promise<boolean> {
    return new Promise(async (resolve) => {
      // open the database
      let result:any = await this._SQLiteService.openDB("test-executeset"); 
      if(result.result) {

        result = await this._SQLiteService.createSyncTable();
        console.log('****** create db ******');
        // create tables
        let sqlcmd: string = `
        BEGIN TRANSACTION;
        CREATE TABLE IF NOT EXISTS users (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users1 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users2 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users3 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users4 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users5 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users6 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users7 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users8 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users9 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users10 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users11 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users12 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users13 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users14 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users15 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users16 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users17 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users18 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users19 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users20 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE TABLE IF NOT EXISTS users21 (
          id INTEGER PRIMARY KEY NOT NULL,
          email TEXT UNIQUE NOT NULL,
          name TEXT,
          FirstName TEXT,
          company TEXT,
          size REAL,
          age INTEGER,
          MobileNumber TEXT
        );
        CREATE INDEX IF NOT EXISTS users_index_name ON users (name);
        CREATE INDEX IF NOT EXISTS users_index_email ON users (email);
        PRAGMA user_version = 1;
        PRAGMA foreign_keys = ON;
        COMMIT TRANSACTION;
        `;
        result = await this._SQLiteService.execute(sqlcmd);
        console.log("result.changes.changes " + result.changes.changes)
        if(result.changes.changes == -1) resolve(false);
        let set: Array<any>  = [
          { statement:"INSERT INTO users (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users1 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users1 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users1 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users1 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users1 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users2 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users2 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users2 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users2 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users2 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users3 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users3 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users3 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users3 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users3 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users4 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users4 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users4 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users4 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users4 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users5 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users5 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users5 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users5 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users5 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users6 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users6 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users6 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users6 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users6 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users7 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users7 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users7 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users7 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users7 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users8 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users8 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users8 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users8 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users8 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users9 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users9 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users9 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users9 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users9 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
          { statement:"INSERT INTO users10 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Simpson","Tom","Simpson@example.com",69,"4405060708"]
          },
          { statement:"INSERT INTO users10 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Jones","David","Jones@example.com",42,"4404030201"]
          },
          { statement:"INSERT INTO users10 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Whiteley","Dave","Whiteley@example.com",45,"4405162732"]
          },
          { statement:"INSERT INTO users10 (name,FirstName,email,age,MobileNumber) VALUES (?,?,?,?,?);",
            values:["Brown","John","Brown@example.com",35,"4405243853"]
          },
          { statement:"UPDATE users10 SET age = ? , MobileNumber = ? WHERE id = ?;",
            values:[51,"4404030202",2]
          },
        ];
        result = await this._SQLiteService.executeSet(set);
        console.log("result.changes.changes " + result.changes.changes)
        if(result.changes.changes != 55) resolve(false);
        resolve(true);
      } else {
        resolve(false);
      } 
    });
  }
Karyfars commented 3 years ago

@jepiqueau thanks for the reply!

im using the current 2.3.0 version. version 2.2.1-3 had the same behavior.

sry i made an error, my estimated query amount was way off. the executeSet call contains between 200 and 850 INSERT OR REPLACE Statements.

i've limited the number of statements in an executeSet to 50, but still the same error. right now i execute between 39 to 66 executeSet calls with each containing 50 statements until it fails.

so it is quite random when it appears.

jepiqueau commented 3 years ago

@Karyfars can you share on github your app or a replicate having the bug

jepiqueau commented 3 years ago

@Karyfars dis you try with iOS?

jepiqueau commented 3 years ago

@Karyfars Can you show me some of the REPLACE Statements, i never test REPLACE

Karyfars commented 3 years ago

@jepiqueau iOS works just fine. it only occures on android.

i am trying to strip down the problem step by step to create a reproducible outcome. but it happens so randomly. i will try to create a new clean app through the weekend where i try to reproduce the error in a shareable environment.

Karyfars commented 3 years ago

@Karyfars Can you show me some of the REPLACE Statements, i never test REPLACE

Here is a debug output which contains some of INSERT OR REPLACE Statements

V/Capacitor: callback: 127015925, pluginId: CapacitorSQLite, methodName: executeSet, methodData: {"set":[{"statement":"\n\t\t\tINSERT OR REPLACE INTO ingredient\n\t\t\t(\n\t\t\t\tidentifier, version, published, name, category\n\t\t\t)\n\t\t\tVALUES\n\t\t\t(\n\t\t\t\t?, ?, ?, ?, ?\n\t\t\t)\n\t\t","values":["BqVfeB3zo16SoJkTB5AnT",1,"2020-06-07T15:49:35.438Z","Weizenmehl zum Bearbeiten ","Legacy"]},{"statement":"\n\t\t\tINSERT OR REPLACE INTO recipe_ingredient_allocation\n\t\t\t(\n\t\t\t\trecipe, ingredient_allocation\n\t\t\t)\n\t\t\tVALUES\n\t\t\t(\n\t\t\t\t?, ?\n\t\t\t)\n\t\t","values":["1oG9VVb7l90nt5OU8JJeMI","3jWy0mXsSqgCEADfrQW3QM"]},{"statement":"\n\t\t\tINSERT OR REPLACE INTO ingredient_allocation\n\t\t\t(\n\t\t\t\tidentifier, version, published, customMeasurement, measurement, measurementImperial, quantity, quantityImperial, ingredient\n\t\t\t)\n\t\t\tVALUES\n\t\t\t(\n\t\t\t\t?, ?, ?, ?, ?, ?, ?, ?, ?\n\t\t\t)\n\t\t","values":["3jWy0mXsSqgCEADfrQW3QM",5,"2020-07-27T13:12:23.933Z","","","",-1,-1,"BqVfeB3zo16SoJkTB5AnT"]}, .....
jepiqueau commented 3 years ago

@Karyfars in iOS did you check the database after completion? is it correct or do you have some INSERT OR REPLACE missing?

Karyfars commented 3 years ago

@Karyfars in iOS did you check the database after completion? is it correct or do you have some INSERT OR REPLACE missing?

as far as i know everything was there on iOS and there is no ticket about missing content from the testing team. but i'll check iOS database, just to be sure.

jepiqueau commented 3 years ago

@Karyfars Is that happen when one statement violates the constraint define in the table schema? can you modify the code of the execSet function in the SQLiteDatabaseHelper as follows and tell me

    public JSObject execSet(JSArray set) throws Exception {
        JSObject retObj = new JSObject();
        // Open the database for writing
        SQLiteDatabase db = null;
        boolean success = true;
        long lastId = Long.valueOf(-1);
        int changes = 0;
        if (set.length() > 0) {
            try {
                db = getConnection(false, secret);
                db.beginTransaction();
                for (int i = 0; i < set.length(); i++) {
                    JSONObject row = set.getJSONObject(i);
                    String statement = row.getString("statement");
                    JSONArray valuesJson = row.getJSONArray("values");
                    JSArray values = new JSArray();
                    for (int j = 0; j < valuesJson.length(); j++) {
                        values.put(valuesJson.get(j));
                    }
                    lastId = prepareSQL(db, statement, values);
                    if (lastId == -1) {
                        success = false;
                        changes = Integer.valueOf(-1);
                        Log.v(TAG, "*** ExecSet failed: breaking lastId -1 ***");
                        break;
                    } else {
                        changes += 1;
                    }
                }
                if (changes > 0) db.setTransactionSuccessful();
            } catch (Exception e) {
                Log.d(TAG, "Error: ExecSet failed: ", e);
                success = false;
            } finally {
                if (!success) {
                    retObj.put("changes", Integer.valueOf(-1));
                } else {
                    db.endTransaction();
                    retObj.put("changes", dbChanges(db));
                    retObj.put("lastId", lastId);
                }
                if (db != null) db.close();
            }
        } else {
            retObj.put("changes", Integer.valueOf(-1));
        }
        return retObj;
    }
jepiqueau commented 3 years ago

@Karyfars if you see some " ExecSet failed: breaking lastId -1 " in the run console means that some of the statements are wrong or infringe some constraints

Karyfars commented 3 years ago

@Karyfars if you see some " ExecSet failed: breaking lastId -1 " in the run console means that some of the statements are wrong or infringe some constraints

@jepiqueau nope sry, there are no constraints set. i've run your code changes and there was no *** ExecSet failed in the console

Karyfars commented 3 years ago

@jepiqueau like i mentioned, when i run the statements through run instead of buffering them and running with executeSet everything runs fine (except it takes way longer).

jepiqueau commented 3 years ago

@Karyfars even just before the crash

Karyfars commented 3 years ago

@jepiqueau my bet at the moment is that the execSet runs just fine. the db.close() causes the exception.

Karyfars commented 3 years ago

@jepiqueau found the cause!

prepareSQL function in SQLiteDatabaseHelper misses a stmt.close() before return lastId; which finalises the statement and the db.close(); in execSet can be called safely.

jepiqueau commented 3 years ago

@Karyfars i miss that one thanks a lot. i will suggest this two changes

    private long prepareSQL(SQLiteDatabase db, String statement, JSArray values) {
        boolean success = true;
        String stmtType = "";
        long lastId = Long.valueOf(-1);
        stmtType = statement.substring(0, 6).toUpperCase();
        SQLiteStatement stmt = db.compileStatement(statement);
        if (values != null && values.length() > 0) {
            // bind the values if any
            stmt.clearBindings();
            try {
                bindValues(stmt, values);
            } catch (JSONException e) {
                Log.d(TAG, "Error: prepareSQL failed: " + e.getMessage());
                success = false;
            }
        }
        if (success) {
            if (stmtType.equals("INSERT")) {
                lastId = stmt.executeInsert();
            } else {
                lastId = Long.valueOf(stmt.executeUpdateDelete());
            }
        }
        stmt.close();
        return lastId;
    }

and

    public JSObject execSet(JSArray set) throws Exception {
        JSObject retObj = new JSObject();
        // Open the database for writing
        SQLiteDatabase db = null;
        boolean success = true;
        long lastId = Long.valueOf(-1);
        int changes = 0;
        if (set.length() > 0) {
            try {
                db = getConnection(false, secret);
                db.beginTransaction();
                for (int i = 0; i < set.length(); i++) {
                    JSONObject row = set.getJSONObject(i);
                    String statement = row.getString("statement");
                    JSONArray valuesJson = row.getJSONArray("values");
                    JSArray values = new JSArray();
                    for (int j = 0; j < valuesJson.length(); j++) {
                        values.put(valuesJson.get(j));
                    }
                    lastId = prepareSQL(db, statement, values);
                    if (lastId == -1) {
                        success = false;
                        changes = Integer.valueOf(-1);
                        Log.v(TAG, "*** breaking lastId -1");
                        break;
                    } else {
                        changes += 1;
                    }
                }
                if (changes > 0) {
                    db.setTransactionSuccessful();
                    retObj.put("changes", dbChanges(db));
                    retObj.put("lastId", lastId);
                    return retObj;
                }
            } catch (Exception e) {
                Log.d(TAG, "Error: ExecSet failed: ", e);
                success = false;
            } finally {
                db.endTransaction();
                if (db != null) db.close();
            }
            retObj.put("changes", Integer.valueOf(-1));
            return retObj;
        } else {
            retObj.put("changes", Integer.valueOf(-1));
            return retObj;
        }
    }
jepiqueau commented 3 years ago

@Karyfars Can you do this and tell me if it works. the changes in execSet are more in line which what we found in the literature. if you revert ok i will implemented in the next release with the support of capacitor 2.4.0 . Thanks for your help in debugging, i was really working in the dark as i could not reproduce it. It will benefit to all the others

Karyfars commented 3 years ago

@jepiqueau your changes work. tested multiple times. so thats an OK from me =D

thanks for your help! have a nice weekend! =)

jepiqueau commented 3 years ago

@Karyfars you are welcome have a nice weekend too

jepiqueau commented 3 years ago

@Karyfars this is now released in 2.4.0 After test could you please close the issue and thanks again

Karyfars commented 3 years ago

@jepiqueau Tested, works! Thank you!