duckdb / postgres_scanner

MIT License
195 stars 35 forks source link

Cannot retrieve big table from postgres #149

Closed hengaini2055 closed 6 months ago

hengaini2055 commented 6 months ago

I use the following statement attach a postgres database

ATTACH 'dbname=postgres host=127.0.0.1 port=5432 user=postgres password=1122334455' as pg_duckdb (type postgres);

I can retrieve a little table such as

select * from pg_duckdb.t1

But when I retrieve a big table (1,000,000), get the errors:

select * from pg_duckdb.t_random
------
SQL ERROR: Invalid Error: Failed to prepare COPY "
    COPY (SELECT "generate_series" FROM "public"."t_random" WHERE ctid BETWEEN '(0,0)'::tid AND '(1000,0)'::tid) TO STDOUT (FORMAT binary);
    ": 
------

I execute this statement from DBeaver 23.3.

OS: windows 11 DBeaver: 23.3.0 duckdb: 0.9.2 Postgres: 15.4

Mytherin commented 6 months ago

Thanks for the report! This is an issue specific to certain clients (in this case the JDBC client). I've pushed a fix in #150. As a work-around you could run SET pg_connection_limit=1.

hengaini2055 commented 6 months ago

image In command ,there are different errors. Does I update the extensions to the newest version?

Mytherin commented 6 months ago

You can install the nightly as follows:

force install postgres_scanner from 'http://nightly-extensions.duckdb.org';