alitrack / duckdb_fdw

DuckDB Foreign Data Wrapper for PostgreSQL
MIT License
272 stars 18 forks source link

Support for duckdb 0.10.x and old releases #38

Open ahuarte47 opened 4 months ago

ahuarte47 commented 4 months ago

This PR adds support for old releases of DuckDB and the new v0.10.x versions.

It checks the existence of ADBC_VERSION_1_1_0 variable (DuckDB v0.10.0) or DUCKDB_MAJOR_VERSION variable (DuckDB v0.10.x https://github.com/duckdb/duckdb/pull/10840/).

In case any of them exist, we have to define a typedef for the new class ErrorData who replaces the old PreservedError.

Tested for v0.9.2 and v0.10.0.

AmebaBrain commented 2 months ago

Am I correctly understand that without this PR merged #45 won't be resolved?

AmebaBrain commented 2 months ago

@ahuarte47 tried to build from your duckdb_fdw-main_9x-10x-support branch. getting error below. looks because of #include "duckdb.hpp" in line 3 in sqlite3_api_wrapper.cpp. in your PR you remove these 2 files.

sqlite3_api_wrapper.cpp:3:10: fatal error: duckdb.hpp: No such file or directory
    3 | #include "duckdb.hpp"
      |          ^~~~~~~~~~~~
compilation terminated.
make: *** [<builtin>: sqlite3_api_wrapper.o] Error 1

p/s I'm not a C/C++ dev. sorry, if saying smth non-related

ahuarte47 commented 2 months ago

@ahuarte47 tried to build from your duckdb_fdw-main_9x-10x-support branch. getting error below. looks because of #include "duckdb.hpp" in line 3 in sqlite3_api_wrapper.cpp. in your PR you remove these 2 files.

sqlite3_api_wrapper.cpp:3:10: fatal error: duckdb.hpp: No such file or directory
    3 | #include "duckdb.hpp"
      |          ^~~~~~~~~~~~
compilation terminated.
make: *** [<builtin>: sqlite3_api_wrapper.o] Error 1

p/s I'm not a C/C++ dev. sorry, if saying smth non-related

You need to download & extract the desired duckdb .zip file release in the folder where you have this repo (Then you will have there the duckdb.hpp & duckdb.h files).

This is the list of actions that I am doing to build the extension:

apt-get install -y -V make gcc g++ "postgresql-server-dev-${POSTGRES_MAJOR_VERSION}" --no-install-recommends
git clone https://github.com/ahuarte47/duckdb_fdw
cd duckdb_fdw
git checkout main_9x-10x-support
wget --no-verbose -c "https://github.com/duckdb/duckdb/releases/download/${LIB_DUCKDB_VERSION}/libduckdb-linux-amd64.zip"
unzip -o -d . libduckdb-linux-amd64.zip
cp libduckdb.so "$(pg_config --libdir)"
make USE_PGXS=1
make install USE_PGXS=1

No tested for 0.10.2, only for 0.9.x & 0.10.1

ahuarte47 commented 2 months ago

Am I correctly understand that without this PR merged #45 won't be resolved?

I think they are different issues

AmebaBrain commented 2 months ago

@ahuarte47 oh, I see now! I was confused by the readme, which highlighted copying of DuckDB package. but the only visible operation with that package was copying a binary *.so lib file into the postgres lib folder. and no single word that that DuckDB package must be extracted into the same folder where duckdb_fdw source files are located! because it requires two header files from the DuckDB archive.

image

I did it and now could confirm that building from your branch with DuckDB v0.10.2 works well! I've created a postgres docker image with duckdb_fdw + DuckDB v0.10.2 lib installed.

@alitrack could you please merge this PR? it's already tested by 2 people. currently I had to build the image from @ahuarte47 branch, and not from your repo. from your repo I was getting an error mentioned in #45

another question. in readme you have specified

duckdb_fdw_version
--------------------
              20300  

but after compilation I'm getting

postgres=# select duckdb_fdw_version();
 duckdb_fdw_version 
--------------------
              20101

which version is correct? also it would be great to have a release published via Releases. I'm maintaining a repo of postgres images with different FDWs installed. and having official releases created for specific FDW simplifies its reference.

ahuarte47 commented 3 weeks ago

FYI; duckdb-fdw is successfully built and working with duckdb v1.0.0 using this PR

AmebaBrain commented 3 weeks ago

FYI; duckdb-fdw is successfully built and working with duckdb v1.0.0 using this PR

@alitrack , any chance to merge this PR? Duckdb released first major 1.0.0 version, and we have confirmed working build for it with this PR.