janbjorge / PgQueuer

PgQueuer is a Python library leveraging PostgreSQL for efficient job queuing.
https://pgqueuer.readthedocs.io/en/latest/index.html
MIT License
211 stars 2 forks source link

Batched dequeue #22

Closed janbjorge closed 1 month ago

janbjorge commented 1 month ago

Improves performance at lower concurrency.

time PGUSER=testuser PGPASSWORD=testpassword PGDATABASE=testdb python3 tools/benchmark.py -t0.1
Concurrecy: 1   Jobs: 1000   JPS: 5.8k
Concurrecy: 2   Jobs: 2000   JPS: 6.8k
Concurrecy: 3   Jobs: 3000   JPS: 6.1k
Concurrecy: 4   Jobs: 4000   JPS: 5.6k
Concurrecy: 5   Jobs: 5000   JPS: 5.1k
PGUSER=testuser PGPASSWORD=testpassword PGDATABASE=testdb python3  -t0.1  1.95s user 0.69s system 83% cpu 3.154 total