Issues with importing dump files

[avinassh]

A user in discord was having issues related to imports, while debugging the issue I noticed a couple of things.

1. My import file was 30MB ish, but the import never completed. I had to cancel it by killing the process, as ctrl + c didn't work. Next I tried with a 10MB file, it was awfully slow
2. Dumps generated from sqlite don't work as they contain BEGIN TRANSACTION. We need to inform the user about this
3. When the imports fail, it dumps the entire file on the terminal. This would be a bad experience

My sample file generated from SQLite:

PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test3 (
  id INTEGER PRIMARY KEY NOT NULL, 
  x REAL NOT NULL,
  y REAL NOT NULL,
  z REAL NOT NULL
);
INSERT INTO test3 VALUES(1,-2600538349684600625.2,-2771290038714697701.0,7164873977611751598.7);
INSERT INTO test3 VALUES(2,3033783454728238737.6,-4758105825240102504.3,-451561723608579690.39);
INSERT INTO test3 VALUES(100000,9119805424044550079.5,-6910344938214794119.3,1939662950113159212.1);
COMMIT;

when I tried on shell:

$ turso db shell champion-arcana < my-small-db-export.sql

Error: failed to execute SQL: PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test3 (
  id INTEGER PRIMARY KEY NOT NULL,
  x REAL NOT NULL,
  y REAL NOT NULL,
  z REAL NOT NULL
);
INSERT INTO test3 VALUES(1,-2600538349684600625.2,-2771290038714697701.0,7164873977611751598.7);
INSERT INTO test3 VALUES(2,3033783454728238737.6,-4758105825240102504.3,-451561723608579690.39);
INSERT INTO test3 VALUES(100000,9119805424044550079.5,-6910344938214794119.3,1939662950113159212.1);
COMMIT;

[CodingDoug]

This might be more appropriately filed on https://github.com/libsql/libsql-shell-go/ since that's the underlying implementation of the shell.

What version of the CLI are you using? Latest right now is 0.80.0. Your sample script works fine for me including the transaction. (interactively transactions don't work, and that I imagine that will not change soon). However, the Turso shell should also generate dump output using a transaction (which it currently does not). I've filed an issue for that: https://github.com/libsql/libsql-shell-go/issues/149

I'll also point out that the error message isn't at all helpful - it is just echoing back the entire contents of the input without saying what's wrong with it. See: https://github.com/libsql/libsql-shell-go/issues/148

[avinassh]

Oops, I was on v0.79.0. Updated to 0.80.0:

1. smaller files import is working now, but for larger files it is still failing. Since it dumps the whole thing, not sure whats the error is
2. The dump has BEGIN TRANSACTION and it works
3. It isn't as slow as earlier

since other issues are being tracked in shell repo, I am closing this.