marcboeker / go-duckdb

go-duckdb provides a database/sql driver for the DuckDB database engine.
MIT License
646 stars 97 forks source link

Cache the validity mask, data pointers, and the chunk size #254

Closed taniabogatsch closed 1 month ago

taniabogatsch commented 1 month ago

Fixes #253.

Significant performance improvement for scans.

Scan benchmark.

PR timings.

SELECT col1.value FROM stress2.main.table2, took 32.725833ms
SELECT col1 FROM stress2.main.table2, took 192.932708ms
SELECT * FROM stress2.main.table2, took 198.002875ms

Additionally, the included benchmark for all types improved slightly.
BenchmarkTypes-10              5     248644567 ns/op

Before.

SELECT col1.value FROM stress2.main.table2, took 236.263375ms
SELECT col1 FROM stress2.main.table2, took 466.651625ms
SELECT * FROM stress2.main.table2, took 460.474834ms

BenchmarkTypes-10              4     286125906 ns/op