sfackler / rust-postgres

Native PostgreSQL driver for the Rust programming language
Apache License 2.0
3.43k stars 436 forks source link

Tokio_Postgres, poor performance for all column’s other than ID. #1159

Closed AbdulWMughal closed 1 month ago

AbdulWMughal commented 2 months ago

Recent benchmarks against all of the known database clients, ORM’s, Query builders crates puts Tokio_Postgres at one of the worst performing for queries (WHERE) using column’s other than id, such as queryable by name. Most likely this is the case for querying using all other possible column’s other than id. This needs to be looked at and fixed as the difference in performance compared to using id, and other crates performance’s of queryable by name and other column’s is enormously poor. Perhaps taking a look at WTX’s approach could help with fixing the poor performance of Tokio_Postgres querying (WHERE) with column’s other than id.

https://github.com/diesel-rs/metrics/

sfackler commented 2 months ago

What client-side difference do you believe exists between queries with WHERE clauses for an "id column" (is that the system-level implicit OID column, or any indexed column, or something else?) and WHERE clauses for any other column type?

sfackler commented 2 months ago

Looking at the source of the benchmarks, this has nothing to do with "queries using columns other than ID", is has to do with extracting values from the returned rows by index or column name: https://github.com/diesel-rs/diesel/blob/master/diesel_bench/benches/postgres_benches.rs#L115-L193.

This is currently done with a linear scan though the columns: https://github.com/sfackler/rust-postgres/blob/master/tokio-postgres/src/row.rs#L69. A hash map from name to index might improve performance but even that would depend on the number of columns in the rows. I would like to see evidence that real world query patterns would see a meaningful net improvement there before making a change however.

However, if you want to maximize your performance with this library, you are fundamentally going to have to use numeric indices.

AbdulWMughal commented 2 months ago

I see thank you for the clarification. I’m not sure if you’re already in contact with the author and maintainer of WTX but he has some really good ideas on this.