neumino / rethinkdbdash

An advanced Node.js driver for RethinkDB with a connection pool, support for streams etc.
MIT License
848 stars 108 forks source link

Reuse connections used for change feeds #365

Open graphicore opened 6 years ago

graphicore commented 6 years ago

This comment https://github.com/neumino/rethinkdbdash/issues/335#issuecomment-304523060 helped me to find a very subtle and hard to find bug in my application.

Rethinkdbdash is using one connection per query. By default it will open up to 1000 connections per server - that is if you have at some time in the past 1000 concurrent queries

My application is requesting a lot of of change feeds (not 1000 but I ran into it because I changed the default). This leads to the problem that my server eventually runs out of "free" connections. Without any notice the service becomes unresponsive, waiting for free connections (bound by change feeds).

I just checked the official rethinkdb driver. Its connection can handle multiple change feeds at the same time, so, there it would be possible to reuse connections.

At least, it would help if the driver could log something to stdout when the pool is fully in use (maybe after a waiting period).