search

Netflix / Hystrix

Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.
24.15k stars 4.71k forks source link

Hystrix and Connection Pooling #1705

Open MikeBleaz opened 7 years ago

MikeBleaz commented 7 years ago

Hi,

Please excuse this - it could well be a dumb newbie question.

We're using Hystrix as a Circuit Breaker between two services. To speed up performance we're using Connection Pooling. We're using the PoolingHttpClientConnectionManager from Apache HttpClient to handle the Connection Pool.

We're also using Hystrix's 'timeoutInMilliseconds' property to catch issues.

The problem we have is that in normal operation it takes c130ms to create a new HTTPS connection, but once pooled the read time on those connections is c12ms.

So, we've had to set our 'timeoutInMilliseconds' threshold to be higher than we'd like so that Hystrix doesn't kick in whenever our Connection Pool decides to create a new connection. Our Non-Functional Requirements want the latency to the service to be nearer 12ms than 130ms.

Are we doing something wrong?

Cheers,

Mike

jjathman commented 7 years ago

No you aren't doing anything wrong, but I don't think you'll be able to achieve your requirement of 12ms latency when establishing an HTTP connection takes more time than that. You can set your timeout to be less than 120ms and deal with a fallback if you want, possibly retrying again and hoping you get an already established connection, but there's no way you can achieve performance faster than it takes to establish the HTTP connection.

Maybe you need a background process which makes sure all pooled connections are valid all the time? This really only works if you are constantly hitting the same route though.