marcboeker / go-duckdb

go-duckdb provides a database/sql driver for the DuckDB database engine.
MIT License
646 stars 97 forks source link

"found unknown state while pending execute" on DuckDB 0.9.0 #124

Closed niger-prequel closed 11 months ago

niger-prequel commented 11 months ago

When running the following SQL

COPY (
    SELECT * FROM benchmarks
) TO '/tmp/prequel/benchmarks.parquet';

Our go program panics with

found unknown state while pending execute: 3

That panic can be found here. My guess is that 0.9.0 introduces a new state with value 3 for which there isn't a constant and switch case.

marcboeker commented 11 months ago

Thanks for bringing this up. The newly introduced state with the value 3 is DUCKDB_PENDING_NO_TASKS_AVAILABLE. There is currently a PR open that addresses the new API (currently DUCKDB_PENDING_NO_TASKS_AVAILABLE is still missing). The problem with 0.9.0 is, that the amalgamation file is too big to be compiled on the open-source Github runners.

Just because I'm curious: have you build 0.9.0 locally?

niger-prequel commented 11 months ago

We actually built it in a docker container on a Github action runner 🤔

marcboeker commented 11 months ago

Open-source runner or paid runner inside an organization? Because the first one is not powerful enough. That's why go-duckdb based on DuckDB 0.9.0 is currently not released yet.

niger-prequel commented 11 months ago

We don't use self hosted runners yet. So its the standard runner, not one of the larger ones. I can confirm we had a build of 0.9.0 succeed yesterday. The build took ~44 minutes. We did our build within a Debian docker container and we only build with a certain set of extensions (JSON, Parquet, ICU). Maybe there is something different about that setup that works within Github Actions default constraints.

marcboeker commented 11 months ago

@niger-prequel go-duckdb v1.5.1 with DuckDB 0.9.1 and a fix for the missing state has just been released. Could you please check if your problem is resolved now?

niger-prequel commented 11 months ago

Yes, I managed to fix it earlier by pinning the go mod to the head of the 0.9.0 branch and we were able to release that smoothly. So I'm fairly sure the official release should be fine.