paradedb / pg_analytics

DuckDB-powered analytics for Postgres
https://paradedb.com
PostgreSQL License
383 stars 15 forks source link

feat: support prepare pushdown #140

Closed kysshsy closed 1 month ago

kysshsy commented 1 month ago

Ticket(s) Closed

What

Why

How

Pushdown PREPARE, EXECUTE and DEALLOCATE statement to duckdb directly.

Tests

kysshsy commented 1 month ago

Note: There is a issue that PostgreSQL will replan the query when certain catalog changes occur, such as changes to the search path or when a table is deleted. In contrast, DuckDB does not replan when the search path is changed. If there are two foreign tables in different schemas and the prepared statements do not specify the schemas, it may lead to ambiguity or errors when referencing the tables.

create foreign table public.t1(xxx);
create foreign table s1.t1(xxx);

prepare q1 as select * from t1; 
execute q1; (query table public.t1);

set search_path to 's1';

execute q1; (query table public.t1 in duckdb, but it should be s1.t1 in Postgres);
philippemnoel commented 1 month ago

Note: There is a issue that PostgreSQL will replan the query when certain catalog changes occur, such as changes to the search path or when a table is deleted. In contrast, DuckDB does not replan when the search path is changed. If there are two foreign tables in different schemas and the prepared statements do not specify the schemas, it may lead to ambiguity or errors when referencing the tables.

create foreign table public.t1(xxx);
create foreign table s1.t1(xxx);

prepare q1 as select * from t1; 
execute q1; (query table public.t1);

set search_path to 's1';

execute q1; (query table public.t1 in duckdb, but it should be s1.t1 in Postgres);

Can we force DuckDB to replan via some API call? We want to imitate the Postgres behaviour as much as possible.

kysshsy commented 1 month ago

Note: There is a issue that PostgreSQL will replan the query when certain catalog changes occur, such as changes to the search path or when a table is deleted. In contrast, DuckDB does not replan when the search path is changed. If there are two foreign tables in different schemas and the prepared statements do not specify the schemas, it may lead to ambiguity or errors when referencing the tables.

create foreign table public.t1(xxx);
create foreign table s1.t1(xxx);

prepare q1 as select * from t1; 
execute q1; (query table public.t1);

set search_path to 's1';

execute q1; (query table public.t1 in duckdb, but it should be s1.t1 in Postgres);

Can we force DuckDB to replan via some API call? We want to imitate the Postgres behaviour as much as possible.

Yes. We could execute prepare in Duckdb again. It will override the plan. I will fix it tomorrow.

philippemnoel commented 1 month ago

Note: There is a issue that PostgreSQL will replan the query when certain catalog changes occur, such as changes to the search path or when a table is deleted. In contrast, DuckDB does not replan when the search path is changed. If there are two foreign tables in different schemas and the prepared statements do not specify the schemas, it may lead to ambiguity or errors when referencing the tables.

create foreign table public.t1(xxx);
create foreign table s1.t1(xxx);

prepare q1 as select * from t1; 
execute q1; (query table public.t1);

set search_path to 's1';

execute q1; (query table public.t1 in duckdb, but it should be s1.t1 in Postgres);

Can we force DuckDB to replan via some API call? We want to imitate the Postgres behaviour as much as possible.

Yes. We could execute prepare in Duckdb again. It will override the plan. I will fix it tomorrow.

Sweet. That sounds wonderful. We want to be as "Postgres idiomatic as possible", so whatever the user experience Postgres users expect is is what we should try to replicate.

kysshsy commented 1 month ago

Note: There is a issue that PostgreSQL will replan the query when certain catalog changes occur, such as changes to the search path or when a table is deleted. In contrast, DuckDB does not replan when the search path is changed. If there are two foreign tables in different schemas and the prepared statements do not specify the schemas, it may lead to ambiguity or errors when referencing the tables.

create foreign table public.t1(xxx);
create foreign table s1.t1(xxx);

prepare q1 as select * from t1; 
execute q1; (query table public.t1);

set search_path to 's1';

execute q1; (query table public.t1 in duckdb, but it should be s1.t1 in Postgres);

Can we force DuckDB to replan via some API call? We want to imitate the Postgres behaviour as much as possible.

Yes. We could execute prepare in Duckdb again. It will override the plan. I will fix it tomorrow.

Sweet. That sounds wonderful. We want to be as "Postgres idiomatic as possible", so whatever the user experience Postgres users expect is is what we should try to replicate.

Yes, I completely agree. I detect changes in the search path and determine whether to replan in DuckDB. It's ready for review. @rebasedming

Weijun-H commented 1 month ago

I will carefully review this PR today or tomorrow.

philippemnoel commented 1 month ago

Thank you <3