search

xo / usql

Universal command-line interface for SQL databases
MIT License
8.81k stars 346 forks source link

Batch insert in \copy to DB destination #458

Open murfffi opened 2 months ago

murfffi commented 2 months ago

Most databases support multi-row INSERT statements so it is possible to speed up \copy by using batches. This issue is discussed in the PR for Clickhouse \copy implementation - https://github.com/xo/usql/pull/457 .

Specifically, Clickhouse doesn't benefit from multi-row inserts within a Go SQL transaction, because the Clickhouse driver implements the Go transactions as batches anyway. All INSERTs in a transactions are written as a single batch always. Furthermore, the Clickhouse Go driver doesn't seem to support including multiple rows in an Exec on a prepared statement within a transaction/batch.

Still this feature can be implemented for other DBs.

murfffi commented 2 months ago

I volunteer to do this after https://github.com/xo/usql/pull/457 .