cloudflare / pingora

A library for building fast, reliable and evolvable network services.
Apache License 2.0
20.3k stars 1.1k forks source link

"on connect" phase for incoming connections #118

Open LessThanGreaterThan opened 4 months ago

LessThanGreaterThan commented 4 months ago

What is the problem your feature solves, or the need it fulfills?

with http/2 you can send multiple requests over the same connection, which means request_filter will be triggered by each request even through its only 1 connection, however if we would like to rate limit on the actual connections being made instead of the individual requests we would need a "on connect" phase

Describe the solution you'd like

add a new phase that only gets called when the connection is actually made and not on a per request basis

Describe alternatives you've considered

N/A, we can't get the source port/IP to match on that, and even then it would be more of a work around then a solution.

andrewhavck commented 4 months ago

There is a connected_to_upstream phase which sounds like the "on connect" phase you're asking for. There is a reused boolean that indicates if the connection is re-used or not.

LessThanGreaterThan commented 4 months ago

There is a connected_to_upstream phase which sounds like the "on connect" phase you're asking for. There is a reused boolean that indicates if the connection is re-used or not.

that doesn't quite align as with a rate limit you would limit before you hit up the upstream no ?

andrewhavck commented 4 months ago

that doesn't quite align as with a rate limit you would limit before you hit up the upstream no ?

I would assume some number of connections will be made to the upstream before we want to apply the rate limit unless we're concerned about load on the proxy itself?

A limitation with a downstream connection rate limit is that it doesn't have visibility into upstream connections. Suppose the downstream requests are over H2 but upstream requests are over HTTP/1.1, this would result in an amplification of upstream connections. Also, we don't support upstream H2 multiplexing yet (it's on the roadmap) so you would see this same amplification for upstream H2 requests today.

andrewhavck commented 4 months ago

Really request_filter still solves this as downstream requests are one to one with upstream connections. That is unless the ask here is to enforce a limit on connections to the proxy itself.

LessThanGreaterThan commented 4 months ago

Really request_filter still solves this as downstream requests are one to one with upstream connections. That is unless the ask here is to enforce a limit on connections to the proxy itself.

yes, it was about limiting connections to the proxy itself