search

citusdata / citus

Distributed PostgreSQL as an extension
https://www.citusdata.com
GNU Affero General Public License v3.0
10.42k stars 662 forks source link

COPY FROM WHERE in pg12 isn't straight forward to support #2878

Open serprex opened 5 years ago

serprex commented 5 years ago

As is, our copy implementation ignores the WHERE

https://github.com/postgres/postgres/commit/31f3817402da736b29014ace470cf70aeb126ac5#diff-2593bfcd8f00a702b24d894bc081f314R863

This mangling of the expression occurs in code we bypass, preferring our own CheckCopyPermissions which doesn't have parseState

CopyState is an opaque pointer for us, so we can't assign copyState->whereClause

Actual filtering happens here: https://github.com/postgres/postgres/blob/31f3817402da736b29014ace470cf70aeb126ac5/src/backend/commands/copy.c#L2728

Bail out is to check for copyStatement->whereClause & throw an error

marcocitus commented 5 years ago

Can we push down the WHERE clause to the worker? It's a lot less efficient, but at least correct.

pykello commented 5 years ago

Can we push down the WHERE clause to the worker? It's a lot less efficient, but at least correct.

I think this makes sense. In the meeting with @serprex and @onderkalaci we decided to postpone implementing this to next release and just error out in 8.4.