holdenmatt / duckdb-wasm-kit

Hooks and utilities to make it easier to use duckdb-wasm in React apps.
https://www.npmjs.com/package/duckdb-wasm-kit
MIT License
96 stars 4 forks source link

How to create a duckdb file to use in the `path` setting? #12

Open bruno-borges-2001 opened 4 months ago

bruno-borges-2001 commented 4 months ago

I want to use a file to persist the data, but I can't because I don't have a pre-existing file to use? Is there a way to create an empty duckdb file or make so it creates one if it does not exist?

initializeDuckDb({
  config: {
    path: `./db.duckdb`,
    query: {
      castBigIntToDouble: true
    }
  }
})
ramonvermeulen commented 4 months ago

You could do that via the duckdb binary, to give an example, as soon as you run the following:

.open <database_name>.duckdb

DuckDB will create a persistent .duckdb file on your local filesystem.

https://duckdb.org/docs/api/cli/overview.html#opening-database-files

The .open command optionally accepts several options, but the final parameter can be used to indicate a path to a persistent database (or where one should be created). The special string :memory: can also be used to open a temporary in-memory database.

holdenmatt commented 3 months ago

One challenge with wasm is it doesn't have a real filesystem. The duckdb-wasm library has its own "virtual filesystem", but AFAIK there aren't great ways to read/write to duckdb files in the browser context.

I believe the path: config works if you can serve a duckdb file from a file server you can access (e.g. in a public folder), but that won't help you persist back.

What I've usually done in the past is persist data as Parquet files myself (either in a browser's IndexedDb or on a server) and then load them into DuckDB-wasm in memory for querying. If you figure out a better approach to using .duckdb files directly, let me know!