Open kou opened 10 months ago
I was wondering if there is an update on this for the performance comparison. And whether any research had gone in to whether this might work with https://github.com/timescale/timescaledb.
I have some more ideas (e.g. internal naive ring buffer implementation may be bottleneck) but I haven't worked on this so much yet.
I haven't tried this with TimescaleDB but it will work because this doesn't care about executed SELECT
.
It's slower than the PostgreSQL protocol.
The followings may be related but we need to look into it:
arrow::RecordBatch
is slow?arrow::RecordBatch
es to usearrow::ipc::RecordBatchWriter()
. We need to copy PostgreSQL data for it. (Not zero-copy.)arrow::RecordBatch
to Apache Arrow C++?SPI_getbinval()
is slow?nocachegetattr()
https://github.com/postgres/postgres/blob/3edc6580c0e27fb8f13322efd255a88d20dda6c2/src/backend/access/common/heaptuple.c#L496-L712 and it's not a short function. Can we shortcut some operations?