Use Structured Field syntax

[reschke]

Given these are new field definitions, why don't they use "structured field" syntax?

• [ ] RateLimit-Remaining see #22
• [ ] RateLimit-Reset
• [ ] RateLimit-Limit

[unleashed]

This has been discussed at https://github.com/ioggstream/draft-polli-ratelimit-headers/issues/35, but in short the decision to use structured fields was postponed until further progress happened both in structured fields and this very document. This was also pointed out at IETF109, so this issue might be a good place to restart the discussion.

[reschke]

FWIW, "structured fields" is about to be publishedas RFC really soon now.

[ioggstream]

I tried at first to model that with sh- but I was not sure of the outcome.

I'll share some thoughts so that we may get some feedback.

The core functionality of the headers should be to simplify ratelimit processing. This means that the following syntax MUST be supported:

ratelimit-limit: 10      # -> int
ratelimit-remaining: 8   # -> int
ratelimit-reset: 8       # -> int

iiuc a suitable sh- notation could be

limit: sh-integer *( OWS "," OWS limit-member )
remaining: sh-integer
reset: sh-integer
limit-member: sh-integer OWS ";" "w" "=" sh-integer

valid values should map to the following data structures. Is that possible?

limit: 10                       -> int
limit: 10, 100;w=5              -> [ 10, [10, {"w": 5}] ]
limit: 10, 100;w=5, 1000; w=200 -> [ 10, [100, {"w": 5}], [1000, {"w": 200}]

invalid values are:

limit: 10, 20, 30                   # it's not clear which
                                    # window applies after the 1st limit
limit: 10, 20; w=10, 30; w=10       # duplicate window values

an sf-list seem to work... the problem is how to specify with a more formal notation that:

• all members MUST be sf-integer
• the first member MUST NOT have parameters,
• other members MUST have the w parameter
• the w parameter MUST BE a sf-integer

from http_sfv import List

for h in ('10', '10; w=60', '10; w=60; comment="ciao", 50; w=600; comment="ciao"'):
  l=List();
  l.parse(h.encode())
  print(l.to_json())

---
[[10, {}]]
[[10, {'w': 60}]]
[[10, {'w': 60, 'comment': 'ciao'}], [50, {'w': 600, 'comment': 'ciao'}]]

[reschke]

There is no formal syntax to do that - just put the constraints into prose.

[ucarion]

As an example of how you can put those constraints into prose, maybe:

https://httpwg.org/http-extensions/draft-ietf-httpbis-cache-header.html#section-2

Can be a start. Also, at the time of writing the spec reads as:

   quota-policy = request-quota; "w" "=" time-window
                  *( OWS ";" OWS quota-comment)
   quota-comment = token "=" (token / quoted-string)

Which has the consequence that the w param must come first. I don't know if that's intentional, but I think it'd be easier to implement the spec if that requirement were loosened.

I'd be happy to push up a PR attempting to implement the suggestions in this comment, if you'd like.

[ioggstream]

@ucarion w-first is intentional. I suggest to define an editorial transition strategy, changing one field definition at a time in this order.

1- specify RateLimit-Remaining with S-H 2- specify RateLimit-Reset with S-H 3- specify RateLimit-Limit with S-H

@unleashed WDYT?

[unleashed]

Sounds good. I think, given some prose will be needed to convey all the requirements, that examples of what valid and invalid fields look like would help the reader understand those better. Your comment could be a good starting place.

Regarding the positional requirement for the window parameter w mentioned by @ucarion, I think the idea was that it should be present to describe the time frame that applies to a quota policy, but I'm not sure there is much benefit in enforcing the order. Do you see any specific advantage?

[ioggstream]

1- switching to S-H we could provide a non-normative validation function to be executed after processing the value;

2- if w is not specified, further parameters should be forbidden. Putting w first makes ABNF more readable and eases validation as you shouldn't look for w inside the header value.

[reschke]

2 - nope. Do not try to enforce things like this in the grammar. Either people ignore it, or they attempt to write a custom parser and likely get things wrong. Just put the requirement into prose.

[unleashed]

2- if w is not specified, further parameters should be forbidden. Putting w first makes ABNF more readable and eases validation as you shouldn't look for w inside the header value.

I suspected as much, but then it is just a matter of easier specification and easier validation via this extra ordering requirement, which in itself I accept as a valid point, but I can also see how that makes for slightly more complex implementations by special casing the ordering.

@ioggstream @ucarion I don't know whether there have been previous similar discussions in other I-Ds about this topic, but if so it would be great if we could locate them to inform a better decision.

[ucarion]

It sounds like there are two discussions happening at once here:

1. Should the document roll its own ABNF, or should it just "import" SFV ABNF rules and then put further constraints in prose?
2. Should the document require the w parameter come first?

I think these are separable issues; the answer to (1) and (2) can happen independently.

I think we should avoid writing ABNF as much as possible, and instead rely on prose to express constraints that SFV doesn't do for us already. Writing our own ABNF has already caused bugs in the specification. For instance, the current ABNF doesn't accept:

10,10; w=5

Because the ABNF rolls its own version of SFV's parameters, and fails to tolerate spaces between ; and w. I assume that's a mistake, because otherwise it means that a generic SFV parser can't be used with this documen.

I also don't think putting rules on whether w goes first does anyone much good. The spec has to go out of its way to say it, and implementors will have to go out of their way to check this.

[ioggstream]

WRT:

1- if we switch to S-F, and the usual way to do it is using prose, let's prose; 2 w, using a SF parser makes it straightforward. I just pass the ball to @unleashed as he's an implementor; we can get in touch with the others too and maybe they could welcome using a SF-parser. In general the advantage of S-F is that you can just leave the field-parsing to a library. see https://www.fastly.com/blog/improve-http-structured-headers

[unleashed]

1. SF and prose sounds ok to me.
2. I think we can make it a requirement that implementors understand the meaning of w rather than requiring its presence or any particular order in relation to other parameters.