Benchmark inconsistencies: sqlc, pgx, pgx_pool, sqlx

[JakubCzarlinski]

Hi,

The insert query for sqlx, pgx, and pgx_pool is defines as:

sqlxInsertBaseSQL   = `INSERT INTO models (name, title, fax, web, age, "right", counter) VALUES `
sqlxInsertValuesSQL = `($1, $2, $3, $4, $5, $6, $7)`
sqlxInsertSQL       = sqlxInsertBaseSQL + sqlxInsertValuesSQL

where sqlxInsertSQL =

INSERT INTO models (name, title, fax, web, age, "right", counter) VALUES ($1, $2, $3, $4, $5, $6, $7)

This is different from the query in queries.sql for sqlc:

-- name: CreateModel :one
INSERT INTO models (NAME, title, fax, web, age, "right", counter)
VALUES ($1, $2, $3, $4, $5, $6, $7)
RETURNING *;

This unfairly causes memory allocations in the sqlc benchmarks. The version of sqlc used is also out of date. Additionally, inside of the sqlc benchmarks, a db.CreateModelParams struct is created on each iteration, unlike in the pgx, pgxpool, and sqlx benchmarks.

Fixing these issues I get the following results:

BEFORE:

Insert
pgx_pool:       12794   86887 ns/op     298 B/op        8 allocs/op
sqlc:           13254   94329 ns/op     1045 B/op       23 allocs/op

AFTER:
Insert
pgx_pool:       16164   71392 ns/op     297 B/op        8 allocs/op
sqlc:           16290   75926 ns/op     298 B/op        8 allocs/op

The deviation in ns/op might be a result of me watching YouTube, but note the B/op and allocs/op. Not sure where the 1B/op diff is coming from. 1 off errors are difficult.

Will clean up my repo and submit a PR.

[JakubCzarlinski]

There's also this in sqlc:

SELECT *
FROM models
WHERE ID > $1
ORDER BY ID
LIMIT $2;

vs this in sqlx, pgx, pgx_pool

sqlxSelectMultiSQL  = `SELECT * FROM models WHERE id > 0 LIMIT 100

where sqlc is unfairly doing more work with ordering and multiple query params.