duckdb / duckdb_spatial

MIT License
461 stars 34 forks source link

DuckDB Crashes when trying to use r-tree spatial index #405

Open DrewScatterday opened 3 days ago

DrewScatterday commented 3 days ago

Hi! Thanks for your hardwork on DuckDB absolutely loving it in my day to day spatial data work.

I think this may be be related to #391 but not sure. I think I found a bug with r-tree indexing. I'm using example code from this article https://duckdb.org/docs/extensions/spatial/r-tree_indexes#example

I'm using node.js 20.11.1, Windows 10, and duckdb 1.1.0

If I run this code, it dies with no error thrown:

const duckdb = require("duckdb");

function queryWithCallback(conn, sql) {
    return new Promise((resolve, reject) => {
        conn.all(sql, (err, result) => {
            if (err) {
                reject(err);
            } else {
                resolve(result);
            }
        });
    });
}

async function main() {
    const db = new duckdb.Database(":memory:");
    var sqlQuery = `INSTALL spatial;
                    LOAD spatial;
                    CREATE TABLE t1 AS SELECT point::GEOMETRY as geom
                    FROM st_generatepoints({min_x: 0, min_y: 0, max_x: 100, max_y: 100}::BOX_2D, 10_000, 1337);
                    CREATE INDEX my_idx ON t1 USING RTREE (geom);
                    SELECT count(*) FROM t1 WHERE ST_Within(geom, ST_MakeEnvelope(45, 45, 65, 65));`
    const rows = await queryWithCallback(db, sqlQuery);
    console.log(rows);
    console.log("🦆 DuckDB initialized 🦆");
}

main();

If I remove the r-tree index, it works with no problem:

const duckdb = require("duckdb");

function queryWithCallback(conn, sql) {
    return new Promise((resolve, reject) => {
        conn.all(sql, (err, result) => {
            if (err) {
                reject(err);
            } else {
                resolve(result);
            }
        });
    });
}

async function main() {
    const db = new duckdb.Database(":memory:");
    var sqlQuery = `INSTALL spatial;
                    LOAD spatial;
                    CREATE TABLE t1 AS SELECT point::GEOMETRY as geom
                    FROM st_generatepoints({min_x: 0, min_y: 0, max_x: 100, max_y: 100}::BOX_2D, 10_000, 1337);
                    SELECT count(*) FROM t1 WHERE ST_Within(geom, ST_MakeEnvelope(45, 45, 65, 65));`
    const rows = await queryWithCallback(db, sqlQuery);
    console.log(rows);
    console.log("🦆 DuckDB initialized 🦆");
}

main();

Output:

[ { 'count_star()': 390n } ]
🦆 DuckDB initialized 🦆

I also tried using nightly build:

const duckdb = require("duckdb");

function queryWithCallback(conn, sql) {
    return new Promise((resolve, reject) => {
        conn.all(sql, (err, result) => {
            if (err) {
                reject(err);
            } else {
                resolve(result);
            }
        });
    });
}

async function main() {
    const db = new duckdb.Database(":memory:");
    var sqlQuery = `FORCE INSTALL spatial FROM core_nightly;
                    LOAD spatial;
                    CREATE TABLE t1 AS SELECT point::GEOMETRY as geom
                    FROM st_generatepoints({min_x: 0, min_y: 0, max_x: 100, max_y: 100}::BOX_2D, 10_000, 1337);
                    CREATE INDEX my_idx ON t1 USING RTREE (geom);
                    SELECT count(*) FROM t1 WHERE ST_Within(geom, ST_MakeEnvelope(45, 45, 65, 65));`
    const rows = await queryWithCallback(db, sqlQuery);
    console.log(rows);
    console.log("🦆 DuckDB initialized 🦆");
}

main();

I get this as the error message:

node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

�☻] {r: Invalid Error: P☺�
  errno: -1,
  code: 'DUCKDB_NODEJS_ERROR',
  errorType: 'Invalid'
}

Node.js v20.11.1

Any ideas, am I missing something?

Maxxen commented 3 days ago

Hi! Thanks for reporting this issue!

My first hunch is that this is related to https://github.com/duckdb/duckdb/issues/13848#issuecomment-2352674000, but the fact that it only crashes when you instantiate the r-tree index is strange... We run the r-tree tests as part of our CI on windows, so maybe it's node related? I'll try to have a look tomorrow.

DrewScatterday commented 3 days ago

Thanks @Maxxen !! Let me know what you find out

I can test on a Ubuntu WSL node js env just to help sanity check

DrewScatterday commented 3 days ago

Code works with no issues on Ubuntu 22.04.02 and same version of node (20.11.1)

So definitely a windows issue and not a rtree bug hmmm

DrewScatterday commented 3 days ago

I tried installing the nightly version of duckdb on winodws with npm install duckdb@next but the cmd window seems to just hang:

[##################] - reify:duckdb: timing reify:audit Completed in 276ms

edit: uninstalled duckdb and then installed with npm install duckdb@next and then still got the same issue

DrewScatterday commented 3 days ago

The duckdb version that gets installed with npm install duckdb@next is "duckdb": "^1.0.1-dev27.0". Was still getting this issue on that version.

I followed the thread in the issue you linked and did the following:

I then still got same issue

I also tried reinstalling spatial extension with FORCE INSTALL spatial FROM core_nightly; but I still get that weird error message I got above