apecloud / myduckserver

MySQL & Postgres Analytics, Reimagined
182 stars 8 forks source link

pg: `FORMAT TEXT` is not supported for `COPY FROM STDIN` #157

Closed NoyException closed 3 days ago

NoyException commented 1 week ago

In pgserver/connection_handler.go, line 654:

    switch copyFrom.Options.CopyFormat {
        case tree.CopyFormatText:
            // NewTextDataLoader is missing here
        case tree.CopyFormatCSV:
            dataLoader, err = NewCsvDataLoader(sqlCtx, h.duckHandler, insertableTable, copyFrom.Columns, &copyFrom.Options)
    }

The text format is ignored, which makes dataLoader still nil after nil check and causes a nil reference error when executing COPY FROM stdin;

fanyang01 commented 1 week ago

This could be fixed by adding fallthrough or using case tree.CopyFormatText, tree.CopyFormatCSV:. Although the TEXT format is not fully compatible with CSV (see https://github.com/duckdb/duckdb/pull/14464 ), for common cases it should work well.

NoyException commented 1 week ago

This could be fixed by adding fallthrough or using case tree.CopyFormatText, tree.CopyFormatCSV:. Although the TEXT format is not fully compatible with CSV (see duckdb/duckdb#14464 ), for common cases it should work well.

I encountered this problem when executing the COPY in pg_dump (see #130 ), and initially tried to add the fallthrough keyword directly. But the txt format in pg_dump is delimited by consecutive Spaces, so maybe I can try to match CSV with \s+ as the delimiter.

fanyang01 commented 1 week ago

But the txt format in pg_dump is delimited by consecutive Spaces

https://www.postgresql.org/docs/current/sql-copy.html#:~:text=this%20might%20cause.-,DELIMITER,-Specifies%20the%20character

Is it tab (\t) but displayed in your editor as 4 spaces?

NoyException commented 1 week ago

Is it tab (\t) but displayed in your editor as 4 spaces?

You're right. It should be a tab. It's just a 16-character column filled with spaces.