search

rethinkdb / horizon

Horizon is a realtime, open-source backend for JavaScript apps.
MIT License
6.78k stars 351 forks source link

Optimization: Open multiple server connections #360

Open danielmewes opened 8 years ago

danielmewes commented 8 years ago

RethinkDB scales better on multi-core servers when it has multiple concurrent client connections, since some of the query processing is bound to a single thread per connection.

We should open multiple concurrent connections automatically to better utilize the available resources.

It would also be nice if multiple RethinkDB servers could be specified, so that connections are distributed across the servers and Horizon automatically fails over if one RethinkDB server goes down.

mfferreira commented 8 years ago

+1

marshall007 commented 8 years ago

It's worth noting that rethinkdbdash has built-in connection pooling and even supports connecting to multiple servers both explicitly and automatically through discovery mode. We might consider just using that since it's literally a drop-in replacement.

In any case, I think adding connection pooling to the official driver itself wouldn't be too much work over implementing something specifically for Horizon.

stellanhaglund commented 8 years ago

Does this mean that its better to create a new connection for every fetch than a global connection?

lördag 14 maj 2016 skrev Marshall Cottrell notifications@github.com:

It's worth noting that rethinkdbdash https://github.com/neumino/rethinkdbdash has built-in connection pooling and even supports connecting to multiple servers both explicitly and automatically through discovery mode. We might consider just using that since it's literally a drop-in replacement.

In any case, I think adding connection pooling to the official driver itself wouldn't be too much work over implementing something specifically for Horizon.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/rethinkdb/horizon/issues/360#issuecomment-219195220

danielmewes commented 8 years ago

Does this mean that its better to create a new connection for every fetch than a global connection?

Definitely not, that would be very inefficient because of connection setup costs (TCP setup and especially authentication). The idea is to keep a pool of connections to RethinkDB open in the backend.