Open jgaskins opened 2 years ago
So sorry @jgaskins I missed this being opened completely until just now!
I think it'd be cool to get pipelining in for sure. I think your approach here of a different interface for it is probably the only sane way to do it. I could never come up with a safe way to auto-pipeline queries.
No exec I think is fine.
If we can't figure out the hang, we can maybe put a limit as to how many queries are in the pipe?
Mechanism to ensure all result sets are consumed, I don’t have a strong opinion on this.
This is a rough sketch for query pipelining support. Pipelined queries in most DBs are implemented as:
Benchmark
Time spent on 100k
SELECT 42
queries (lower is better):That's 55% less wall-clock time and 58% less CPU time with
localhost
latency. For databases where latency might be in the 1-2ms range, the difference will probably be orders of magnitude. In fact, it might've been a more dramatic difference but I had to split the pipelines into chunks — see "known issues" below.Usage
Via
crystal-lang/crystal-db
:The API I had in mind with this is that
query_one
,query_all
, andquery_each
becomequery
inside thepipeline
block andread_one
,read_all
, andread_each
when you're consuming results.Known issues/Remaining work to do
exec
support10_000.times { pipe.query "..." }
[ ] Mechanism to ensure all result sets are consumed
PG::Connection
?Example:
Closes #155