search

palantir / conjure-java-runtime

Opinionated libraries for HTTP&JSON-based RPC using Dialogue, Feign, OkHttp as clients and Jetty/Jersey as servers
Apache License 2.0
80 stars 95 forks source link

Introduce backpressure when rate limits are hit #993

Open ellisjoe opened 5 years ago

ellisjoe commented 5 years ago

What happened?

Currently a client is able to enqueue an unbounded number of requests into the Dispatcher.

What did you want to happen?

One a certain number of requests are enqueued, we should either start throwing 429s back out of the client, or start blocking the threads enqueueing requests to introduce some form of backpressure upstream.

j-baker commented 5 years ago

I don't believe the client is able to do this without the concurrency limit rising to a higher number than the per-host limit. If you'd like, we should set a 'max concurrency' on the concurrency limiter, which at least avoids the noisy neighbour states?

ellisjoe commented 5 years ago

This happens when the incoming request rate exceeds the concurrency limit, regardless of where the limit is in relation to the per-host limit. This won't prevent the concurrency limit from exceeding the per-host limit, but it'll push back on clients when that does happen.

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not been touched in the last 60 days. Please comment if you'd like to keep it open, otherwise it'll be closed in 7 days time.