electric-sql / pglite

Lightweight WASM Postgres with real-time, reactive bindings.
https://pglite.dev
Apache License 2.0
9.38k stars 202 forks source link

loadDataDir option throws error for gzip file #402

Open davepar opened 3 weeks ago

davepar commented 3 weeks ago

In Node v22, when I try to load an in memory database with {loadDataDir: ) where the file is a gzip file created from .dumpDataDir('gzip'), I get Error: File is corrupted.

When I unzip the file on the command line into a directory, and then load that directory with loadDataDir, it works fine.

If that's a known limitation, it would be nice to have it documented. Thanks!

To reproduce:

const db = new PGlite({ dataDir: databaseFilename, extensions: { cube, earthdistance } });
await db.waitReady;
// Add some tables and records
const savedDb = await db.dumpDataDir('gzip');  // Using 'none' instead works fine
writeFileSync(archiveFilename, Buffer.from(await savedDb.arrayBuffer()));
db.close();

And then:

const loadDataDir = new Blob([readFileSync(archiveFilename)]);
const db = new PGlite({ loadDataDir, extensions: { cube, earthdistance } });
samwillis commented 3 weeks ago

Thanks for the report. This isn't a know limitation, and as far as I'm aware should work.

0xOlias commented 3 weeks ago

Reproduced with Node 18-22 and Bun

https://github.com/0xOlias/pglite-gzip-repro

kevin-on commented 2 weeks ago

I resolved this issue by using new Blob([fileBuffer], { type: 'application/x-gzip' }) to set the Blob type.