timescale / promscale

[DEPRECATED] Promscale is a unified metric and trace observability backend for Prometheus, Jaeger and OpenTelemetry built on PostgreSQL and TimescaleDB.
https://www.timescale.com/promscale
Apache License 2.0
1.33k stars 169 forks source link

Custom COPY with only one DB roundtrip #1814

Closed alejandrodnm closed 1 year ago

alejandrodnm commented 1 year ago

Description

Add custom CopyFrom command that avoids an extra roundtrip

The CopyFrom command from pgx requires the OIDs of the data types is going to insert when constructing the binary message that contains the data.

To retrieve the OIDs a statement is prepared per COPY operation. In our case the statements are very simmilar:

copy {tableName} ( `time`, `value`, `series_id` ) from stdin binary

These statements can be cached, the problem is that we will have a cache entry per table name.

Since we always insert to the same columns and we know the OIDs of the columns we are going to be inserting to, we can avoid the extra roundtrip and the cache by manually passing the OIDs.

This custom copy command is basically a copy of the pgx CopyCommand that accepts the list of OIDs.

Merge requirements

Please take into account the following non-code changes that you may need to make with your PR: