search

megastef / node-crate

Node.js base DB-Driver for CRATE (www.crate.io)
52 stars 18 forks source link

Master #18

Open waysay opened 7 years ago

waysay commented 7 years ago

Added a few extra request handlers to avoid connection timeouts with http and https requests.

megastef commented 7 years ago

Are you able to solve conflicts?

I've got a few thoughts about this issue. Please note JAVA (crate db) might start 100 java threads when nodejs uses 100 sockets. This might kill your crate instance (and gets you more http timeouts ..). I have seen many times that Java servers (e.g. Elasticsearch) got overloaded or started to reject connections when hitting thread limits. Now imagine 100 clients using this lib connect with 100 sockets ... it would be nice to make maxSockets configurable e.g. via an environment variable. We could set a new httpAgent https://nodejs.org/api/http.html#http_new_agent_options, instead of modifying global httpAgent settings (because other parts of the node program might need more sockets for other backends ...). In a long run might be better to queue requests (with a limited number of parallel actions to keep Java side healthy).