duckdb / sqlite_scanner

DuckDB extension to read and write to SQLite databases
MIT License
189 stars 19 forks source link

db Loads in Windows but not Linux #100

Open AimSmall37 opened 3 weeks ago

AimSmall37 commented 3 weeks ago

What happens?

Using DuckDB CLI, both v0.10.3 70fd6a8a24 and sqlite_scanner version 50b7870.

On Linux, receive file error. Exact same file (16GB) on Windows loads.

D ATTACH '~/DataAnalysis/Research/DNAGedcom.db'as dna (TYPE SQLITE); D use dna; Invalid Error: Unable to open database "~/DataAnalysis/Research/DNAGedcom.db": unable to open database file

PS D:\Program Files (x86)\Incline Software\Ancestral Quest 14\DNA\Software\DNAGedcom> duckdb 'DNAGedcom.db' v0.10.3 70fd6a8a24 Enter ".help" for usage hints. D .tables

Posted the issue on Discord in the DuckDB server, extensions channel. User carlop suggested I post the issue here.

https://discord.com/channels/909674491309850675/1209495191380172862/1246457421178605709

To Reproduce

PS D:\Program Files (x86)\Incline Software\Ancestral Quest 14\DNA\Software\DNAGedcom> duckdb 'DNAGedcom.db'

In Linux: D ATTACH '~/DataAnalysis/Research/DNAGedcom.db'as dna (TYPE SQLITE); D use dna;

OS:

Linux Mint 21.3 x86_64 Kernel: 5.15.0-107-generic

SQLite Version:

3.33.0

DuckDB Version:

v0.10.3 70fd6a8a24

DuckDB Client:

CLI

Full Name:

Patrick Logan

Affiliation:

Personal Use

Have you tried this on the latest main branch?

Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?

AimSmall37 commented 3 weeks ago

Tried on another VM that is similar with a different technique... same result.

wilbur@bigdataprox:~/duckdb$ ./duckdb '~/DataAnalysis/Research/DNAGedcom.db' v0.10.3 70fd6a8a24 Enter ".help" for usage hints. D .tables terminate called after throwing an instance of 'std::runtime_error' what(): Unable to open database "~/DataAnalysis/Research/DNAGedcom.db": unable to open database file Aborted (core dumped)

AimSmall37 commented 3 weeks ago

wilbur@bigdataprox:~/duckdb$ ./duckdb v0.10.3 70fd6a8a24 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. D SELECT extension_name, extension_version FROM duckdb_extensions() WHERE extension_name == 'sqlite_scanner'; ┌────────────────┬───────────────────┐ │ extension_name │ extension_version │ │ varchar │ varchar │ ├────────────────┼───────────────────┤ │ sqlite_scanner │ 50b7870 │ └────────────────┴───────────────────┘ D

AimSmall37 commented 3 weeks ago

DNAGedcom.db: SQLite 3.x database, last written using SQLite version 3037002, writer version 2, read version 2, file counter 684083, database pages 4486425, cookie 0x1c2, schema 4, UTF-8, version-valid-for 684083

AimSmall37 commented 3 weeks ago

Ok... update here. Found an older version of the database. It opened in Linux in the same environment. Not really sure what's that different between them.

Older db that worked. DNAGedcom.db: SQLite 3.x database, last written using SQLite version 3015002, file counter 85754, database pages 23068, cookie 0x2a, schema 4, UTF-8, version-valid-for 85754

db that did NOT work (apparently it's been modified slightly from the same file I had in Windows). DNAGedcom.db: SQLite 3.x database, last written using SQLite version 3037002, writer version 2, read version 2, file counter 684083, database pages 3949789, 1st free page 3406561, free pages 28, cookie 0x1c1, schema 4, UTF-8, version-valid-for 684083

Windows db version that worked DNAGedcom.db: SQLite 3.x database, last written using SQLite version 3037002, writer version 2, read version 2, file counter 684083, database pages 4486425, cookie 0x1c2, schema 4, UTF-8, version-valid-for 684083

AimSmall37 commented 3 weeks ago

Had the original compressed. I uncompressed it and it worked in Linux. Can only guess a modification to it did something to it, but SQLite had no issue using it.

Here is the fresh restore file info. DNAGedcom.db: SQLite 3.x database, last written using SQLite version 3026000, writer version 2, read version 2, file counter 684082, database pages 3949789, 1st free page 3406561, free pages 1, cookie 0x1c1, schema 4, UTF-8, version-valid-for 684082