search

hyperium / tonic

A native gRPC client & server implementation with async/await support.
https://docs.rs/tonic
MIT License
10k stars 1.01k forks source link

Include Tower feature 'load-shed' #769

Open GrandmasterTash opened 3 years ago

GrandmasterTash commented 3 years ago

Feature Request

Please add the tower feature 'load-shed' to cargo.toml

Motivation

Under heavy load I'm seeing thread counts and memory explode (for some reason when the clients begin timing out).

This isn't prohibited by tower::limit::ConcurrencyLimitLayer as it backs-up the requests behind a semaphore.

I'm also not able to try the rate-limit layer because I'm using serve_with_shutdown() which requires layers implement Clone.

I would like to try the Tower feature tower::load_shed::LoadShed to see if this can alleviate my problem.

Proposal

Amend the dependency in Cargo.toml for tower to include "load-shed" in list of features.

LucioFranco commented 3 years ago

@GrandmasterTash if I understand correctly you'd like to use a time based rate limiter than a concurrency one? I think that makes sense. I think the rate limit layer currently is quite limited, we should improve this for sure.

GrandmasterTash commented 3 years ago

Pretty-much yes. I actually had this layer in-mind - but the rate limit layer could be useful as well.