search

duckdb / pg_duckdb

DuckDB-powered Postgres for high performance apps & analytics.
MIT License
1.61k stars 56 forks source link

Tests for COPY and COPY to local parquet file #265

Open JelteF opened 1 month ago

JelteF commented 1 month ago

We completely broke COPY support without us noticing. That problem is being fixed by #264, but it might happen again if we don't add some tests. For the 0.1.0 release we can manage with manual testing though.

I think the easiest way would be to have a test write locally to parquet and read the resulting file to see that it contains the expected content in parquet format. We currently don't support writing parquet locally but that should be fairly easy to do forcing the COPY to go through DuckDB if it contains a .parquet suffix.

mkaruza commented 1 month ago

For testing we should check situations where filename part of COPY statement match table name or column name. For example this should work with COPY (from COPY .. TO remote perspective):

COPY (SELECT * FROM "s3://remote/csv") TO 's3://remote/csv' (FORMAT CSV, HEADER, FORCE_QUOTE ('s3://remote/csv'));

Where s3://remote/csv is table and/or column name.