Throttling scope

[ioggstream]

I expect

A way to define a throttling scope.

This is related to the scope of Retry-After https://github.com/httpwg/http-core/issues/99#issuecomment-517803876 and should be fixed after that.

Related discussion on the Sunset header: https://tools.ietf.org/html/rfc8594#section-1.4

Note

The scope for Retry-After is not going to be refined https://github.com/httpwg/http-core/issues/99#issuecomment-517803876

but Roy Fieldings suggested to consider a RateLimit-Scope header eg. https://github.com/httpwg/http-core/pull/317#issuecomment-585868767 instead of a scope parameter.

@unleashed I stubbed a draft for investigating on retry-scope that can be useful to better address the ratelimit-scope. PTAL to the preliminary feedback here https://github.com/ioggstream/draft-polli-retry-scope/issues/1

[guzi99]

Any update on this issue?

I have the same feedback. A calling app can represent multiple identities, such as an user in a tenant, where both the user and the tenant can have their own rate limits. How can we express the different scope of rate limits?

Besides, it would be helpful to reconcile and look at the retry-after scope together, since they are both about rate limiting and throttling, but different state(every request vs. throttled request).

[ioggstream]

@guzi99 we hoped to solve this issue with the retry-scope but it seems that I-D did not get enough support...

maybe @unleashed can provide some more hints on the subject.

[unleashed]

@guzi99 I hear you - we share the use case. I commented on the linked issue, but at this stage I think the scoping is a very broad and nuanced topic that deserves its own discussion and process separate from the current draft to avoid stalling its progress indefinitely, yet at the same time we should be careful to make it forward compatible with whatever we can come out with for scoping - we currently try to do that by avoiding guarantees around scoping and future requests.

At this point implementors that want to be more explicit about scoping will need to rely on quota comments and/or additional semantics provided by other headers.

Discussion on how the scoping should be conveyed to clients and intermediaries should still happen at some point, perhaps in connection with the broader HTTP WG community.

[wayne2eng]

@unleashed based on your comment, we propose to use quota-comment custom parameter to pass the scope information until there is a formal support for scoping in the future version of RateLimit headers standard.

An example is 100;w=300;scope="application";comment="request count limit for 5 minutes"

Do you see any issue in this approach? Note that, for now, the value strings of scope token could be anything relevant to the quota limit policy for instance user, application, mailbox, etc.

[unleashed]

@wayne2eng no, I don't see any problem as long as people know to use some specific parameter name - "scope" sounds ok, but once it is used as a free form string in the wild it will have to stay that way forever, so maybe we want to refer to it explicitly in the text, much like "w", so as to reserve this specific usage.

In fact, given how we essentially avoid the question of scope, acknowledging people will want to provide this information and setting up a particular parameter name is better than just letting them figure out a free form parameter name which will very likely be different for each implementer.

[unleashed]

Suggested parameter name: s - wdyt?

[guzi99]

Sounds good to me.

[wayne2eng]

Agreed. I see no issue to use "s" as the keyword for the scope usages

[ioggstream]

@wayne2eng @unleashed Instead of a taxonomy of scopes, my proposal here https://github.com/httpwg/http-core/pull/317#issuecomment-585634120 is based on URLs. Ideas?

[unleashed]

I think it's a starting point - and with that I mean that it certainly covers in a very straightforward fashion a significant chunk of use cases. However it also limits those to paths/resources of the service and leaves out other factors that might limit or expand the scope (ie. anything external or environmental, like specific network address or server instance, or only odd minutes in the wall clock, or more complex cases like specific URI patterns).

While having an common use case covered is a win IMO even if it's using a simple form, I think we might still want to add a parameter specifically to mean: "here's some scope information we can't cover via other means, so it's for you to pick it up and interpret it". If that's not s because we are going to use it as in your proposal (which I would be in favor of!), then we might want to still pick another one for the "expanded scope information" use case.

How does this sound?

[ioggstream]

it also limits those to paths/resources of the service and leaves out other factors

yes, it's very HTTP-ish and simplistic. but if we want to do something quick and clear, then I think URL is the only possible choice I can think of. In general I'm quite neutral on that. Another choice can be to use custom strings/label: but then the semantic will not be clear.

I'm fine on addressing scope here only if that won't delay the publication too much.

Since @unleashed you are the implementation expert here, how would you design that?

[wayne2eng]

@ioggstream and @unleashed , I had read the retry-scope draft early on. I also came out the realization that retry-scope covers the target (or destinate) entities or scopes like subpath of a url. This target-oriented scope would work in the case like mailboxes or drivers when those can be expressed within a uri.

However, we do have scenarios in which the quotas are applied to requestors (request originators) like users or applications. For those types of scope, the current retry-scope proposal doesn't have coverage.

btw, in my opinion, the values of scope for each service could be very specific. For example, in email services, mailbox or user may be the scope entities. In other generic services, the throttle scope could be users, calling applications/services, tenancies, etc.

[ioggstream]

@wayne2eng could you please share some examples of the expected header values?

It would be great to see some practical values that can be useful for your use cases.

[wayne2eng]

Here are a few examples in our scenario.

Example of resource unit quota warning of an application running in one tenancy RateLimit-Limit: 6250, 6250;w=300;s="application|tenant";comment="Application Resource Unit 5-minute Quota" RateLimit-Remaining: 812

Example of user egress quota warning RateLimit-Limit: 100000000, 100000000;w=3600;s="user";comment="User Egress KB One Hour Quota" RateLimit-Remaining: 147

Note that scope is used as hints to the caller. There was any actual id returned back to them. We expect that the callers to track those entities when making requests. When throttles occur, they can figure out which entity breaches or is about to breach the limit.

[unleashed]

@wayne2eng this is what I tried to express when saying that s as a URI reference leaves out some complex use cases - that said, it might be just enough for a significant set of common and simple use cases. What I like about @ioggstream's suggestion is that these simple use cases can be easily addressed, which IMO helps adoption. Those cases where the s parameter as suggested cannot work will require some extra information that the standard will not be able to help with (at least not at this stage, and that may be a matter that requires a different I-D so that scopes in this and other contexts like Retry-After can reference a set of well-defined semantics, which in itself is something that not even API management vendors seem to agree upon).

For this extra information that cannot fit the simple definition of an URI reference I suggest we use a standardized, service-specific scope parameter (ie. think vendor-specific extensions) which clients and intermediaries will just ignore if they can't possibly understand it, while clients with service-specific knowledge can leverage that information to adjust their rates in a smarter way.

[wayne2eng]

I am interchanging to use scope and entity term. IMO, a throttle entity is same as the scope we are talk about here.

The concern I have in URI reference only scope is that there will be many really basic scopes/entities left out. For example, user entity relying on OAuth token, App entity based on app registration token id, etc. Those entities like request originators cannot be easily expressed in a URI reference. But they are critical part of the quota policy definition.

I agreed that the s parameter shouldn't become too complex. This "pipeline" notion in the examples I put early could be one of those complexities which could be avoid. We will reexamine the necessity of that. However, we should have a standardized way to express most common scopes/entities like user, application or tenancy.

If custom response header fields are what you meant by vendor-specific extensions for instance x-ratelimit-scope, we have thought about that too. However, those custom fields may not be propagated appropriately across proxies or middle-layer services.

[wayne2eng]

@unleashed and @ioggstream , discussed with the team again. We found out that, in the most of our scenarios, we really just need to communicate back to the calling servers about their auth-token based scope/entity besides the uri reference proposed in Retry-Scope. In essence, there are only two tokens - user auth token and app auth token. We think that we should have those two basic auth token defined in the built-in scope support.

Any thoughts on this?

[ioggstream]

So when you refer to scope, you mean oauth scope, right?

[wayne2eng]

Let me clarify this a bit.

What I meant by scope above was still the scope concept proposed in rate-limit and retry-scope drafts. for example, the value of "s=" parameter. As we discussed early, uri scope value based has been introduced in retry-scope. And I assume that uri scope will be supported in rate-limit (yes, we do need the scope in rate-limit since retry-scope only gets set back when 429 code is sent).

Besides uri based scope support, I suggest supporting the scope value to indicate the entity type based on the auth types. This helps us to tell the callers a rate-limit numbers returned due to limit approaching for the current authenticated user or the current authenticated application. Then the callers could take some actions to slow down the requests accordingly like slowing down the request from the user account etc. App auth-based scope would be similar. Usually, the service authenticates a request from an app based on the app registration model. If the service has a quota policy applied to the app as a whole, it can return rate-limit and indicate the limit is meant for the app itself but not individual users.

[ioggstream]

This is a complex slope: for this reason we stated auth is off-scope

Authorization: : RateLimit fields are not meant to support authorization or other kinds of access controls.

Supporting in the spec something related to auth means having a thorough security review... I won't conflate URL and oauth-scope parameters for example. It will be complex to parse and prone to bad implementation, if you use it for actually shaping the traffic.

I suggest you to use a myscope parameter for that, and share your results with the httpapi workgroup: if that works we could register it.

I am not sure whether it would be better to pass this auth-related info in the ratelimit or in another header .

[wayne2eng]

@ioggstream, no, what I meant by auth-base scope isn't really authentication. Let me back to the beginning. What we need was to send back a signal about the target entity types which various policies apply quota on. Which includes users or apps registered in particular tenant. It just happens to be those two types, which I mentioned - user or app, are related to auth-token.

Note that we also recognized that passing actual token or any id will cause security or PII concern. Instead, just simple types which are commonly used in many throttling policies. But we let the caller that it's the user, which he just sent request, is hitting the rate-limit. And we rely on the caller to find the exact user id on his side.

Here is the thing. If the committee thinks that "s=" parameter will only support uri reference in v1 (please confirm that), then we will rethink that our original plan to expose the common types like user or app off the custom header field. IMO, this approach could confuse developers in the long term. For some throttle entity targets, developers need to check "s=". For other entities which could be commonly used in throttle policies, they have to check the custom headers.

[wayne2eng]

@unleashed and @ioggstream , any thoughts on the previous comments I posted? We have to make a decision for either utilizing the standard "s=" parameter notion or using the custom header field for this throttle scope value.

[unleashed]

@wayne2eng I will delegate to @ioggstream on whether the s parameter could adopt this use case, although I suspect that the discussion should consider whether it applies as well to Retry-After's scope.

However, assuming s is out of the question, I'd like to suggest a specific parameter name for custom scope information. It does not matter what the name is as long as it is standardized and clearly conveys that it contains information intended for custom clients that will extract and process it, the idea being that we don't have a proliferation of parameter names in the wild used to satisfy custom needs.

[ioggstream]

Sorry for the late reply, and Hap.py New Year everybody! :tada:

Re-reading the I-D I noted that the concept of scope discussed here is resource related . This is originated by the fact that we wanted to provide a solution that is based on HTTP concepts to be as interoperable as possible.

@wayne2eng let's decouple the I-D from this specific issue:

1. feel free to experiment with any parameter in your ecosystem;
2. in 6 months, present your results to the WG so we can discuss adding new parameters to the registry. Consider that if you're using s it might be that the WG could rename the parameter.

I suggest supporting the scope value to indicate the entity type based on the auth type IMHO your use case is cool, but is too specific to be widely implemented in an interoperable way (eg. we should define what's an "entity type", what's an "auth type", how they should be (de)serialized, their semantics, ...).

@unleashed

suggest a specific parameter name for custom scope information

Maybe we can suggest naming conventions for parameters (eg. rht-scope, msft-scope, rht-s, msft-s) but if "scope" is not

the given resource-target, its parent path or the whole Origin

we should define elsewhere what "scope" is.

[unleashed]

@ioggstream I was thinking one (unique) parameter name for custom/vendor-/service-specific scope information could be enough, but your idea could also be fine, so not feeling strong about this as long as we don't end up with a situation in which clients can't tell these custom parameters apart.

[ioggstream]

@unleashed question: if we define e.g. x-s without a semantic, could a proxy/gateway change its content causing interoperability issues?

Eg with a single parameter

Server returns:

RateLimit-Limit: 10, 10; w=10; x-s="user-scope"

Proxy replaces:

RateLimit-Limit: 10, 10; w=10; x-s="/foo/bar"

Eg with a custom parameter

Server returns:

RateLimit-Limit: 10, 10; w=10; msft-s="user-scope"

Proxy replaces:

RateLimit-Limit: 10, 10; w=10; rht-s="/foo/bar"; msft-s="user-scope

[unleashed]

@ioggstream if the proxy would do that then it should have service-specific knowledge, which then makes the case that it should be allowed for a proxy to replace, or, if anything, append to that information, because, well, it must know what it is doing.

Indeed with vendor-specific parameters you could have more information, but I wonder whether that would actually be used in the real world? I would have thought that if someone really needs to make use of this custom parameter, then they can encode a number of values within, as long as it all belongs to this parameter and doesn't interfere with parsing. (the rules for parsing would then be more complex, which is why you'd suggest using multiple parameters following a pattern?)

[ioggstream]

@unleashed

if the proxy would do that then it should have service-specific knowledge...if anything, append to that information

sure, but even appending the information results in something ugly x-s="user-scope /foo/bar"... my experience is that seldom vendors are interested in interoperating with other products.

I wonder whether that would actually be used in the real world

this is the reason I think we should release the spec and see what happens in the wild before registering parameters.

if someone really needs to make use of this custom parameter, then they can encode a number of values within

if we provide this functionality, shouldn't we need to define syntax and semantics to avoid implementers to stuff anything in it?

why you'd suggest using multiple parameters following a pattern

my idea is that once we suggest a way to avoid parameter clashes the parsing will be easy. Since it's a SF, parameters will be deserialized as a dictionary, eg.

from http_sfv import List
l = List()
l.parse( b'10,   10; w=10; rht-s="/foo/bar"; msft-s="user-scope"; burst=1000' )
quota = l[1]
print(quota.__dict__)
{ 
  'value': 10, 
  'params': {
    'w': 10, 
    'burst': 1000, 
    'rht-s': '/foo/bar', 
    'msft-s': 'user-scope'
  }
}

[wayne2eng]

@ioggstream , thanks for the code sample! I actually like this design. However, there are just a few questions I am not clear about.

1. What is this rht-s: "/foo/bar"? Is this uri-based scope?
2. Why there would be two scopes returned in a single rate-limit response header in your sample? For instance, you had both rht-s and msft-s. What was the scenario you were envisioning?

[ioggstream]

@wayne2eng imagine the following setup:

1. an on-premise red-hat api gateway using rht-s which is an URI (absolute or relative, we don't care).
2. a cloud WAF using msft-s taking care of authnz using azure functionalities
3. the following setup:

client --> Azure WAF (user checks, DoS protection, ... ) --> on premise RHT API Gateway --> Internal API

prefixing parameters, we support infrastructures with different gateways for different purposes, each one with its own semantic.

Clearly rht-s and msft-s could apply to different quota-policies too, eg.

10; w=10, rht-s="/foo/bar", 2600, w=3600, msft-s="user-scope"

[ioggstream]

@wayne2eng PTAL #83

• summarize this discussion in the FAQ
• mention extension mechanisms such as parameter registry

cc: @unleashed I'd close then.

[wayne2eng]

Thanks for the update, @ioggstream!