search

timgit / pg-boss

Queueing jobs in Postgres from Node.js like a boss
MIT License
2.13k stars 158 forks source link

Good pool size? #49

Closed terehov closed 6 years ago

terehov commented 6 years ago

Awesome library! We‘re about to integrate it into our Postgre-centric Node.js Framework.

Since most hosted Postgres solutions have a connection limit, I am trying to figure out a good configuration setup. What are your experiences with the connection pool. How many connections is the minimum? Would it theoretically work with one? What are the drawbacks? Thanks!

cheers

Eugene

timgit commented 6 years ago

Eugene, pg-boss has a default pool size of 10 connections. Postgres normally ships with max connections set to 100, but I’m not sure what the various limits are for hosted solutions out there. I would say try the defaults first and tune as needed.

Thanks!

terehov commented 6 years ago

Hi Tim,

thanks! Yeah, I know that you use the default pg pool size. But as soon as there are multiple node instances running it sums up pretty fast. Thats why we have a build in functionality that manages the pool size across all the nodes and automatically, gracefully adjusts it to reflect the number of nodes, to enable an horizontal auto-scaling scenario (e.g. for lambda functions). Thats why I need to know, what the minimum number of connections could be, to add this information to our pool-logic. Would it "theoretically" also work with only one open connection, or do you need at least two, in order to run stuff in parallel?

Thanks!

timgit commented 6 years ago

I haven’t seen anything in the pg pool docs that imply a minimum, so as long as your volume is low enough per instance, you could probably get pretty far with a single connection in the pool. The best advice I can give you is to measure it for your load. Hope this helps.

terehov commented 6 years ago

Yes, Thanks!