ietf-wg-httpapi / rfc7807bis

Revision of RFC7807: HTTP Problem Details
Other
20 stars 8 forks source link

Forward compatibility #36

Closed awwright closed 2 years ago

awwright commented 2 years ago

(In reply to #32, which I think is blocked by this issue.)

I would call this specification "forward compatible" if we can add new features to it without interfering with existing compliant implementations. This is important so that standards can evolve to support new functionality, like proposed in #32.

It does not appear to me that there's any mechanism for forward compatibility, and this is noted:

Note that because extensions are effectively put into a namespace by the problem type, it is not possible to define new "standard" members without defining a new media type.

While the media type can be "extended" in some fashion, this is not the same as forward compatibility, because all extensions are by definition user extensions, so they cannot be standardized. I.e. there is no way to define new "global" members.

I think this is important to fix. I see a few options:

(1) Note how the concept of "user" extensions has some amount of ambiguity. In order to know what a non-standard member means, you have to know who issued the Problem document. But there's no standard way to encode this! (It does say the type is "effectively" a namespace for extensions, but this doesn't seem like particularly authoritative language.) Applications that are tightly coupled with a specific API may not have trouble consuming user extensions, but other user-agents (e.g. spiders) may have a problem deciphering them. Because of this, I think we could more rigorously specify the behavior without significant breakage "in the wild". For example, specify that extension members must use a URI, and all other members have a standard definition. Or, choose some prefix that sorts high in ASCII, like ! or $, to indicate standard "global" members.

(2) If this still causes too much breakage, we could either reserve the member names that would cause breakage, or I would support minting a new media type.

We could also consider defining a JSON-LD vocabulary. There's likely a good way to encode the problem details that's easily detectable as a document describing the error, and having a standardized form and properties, while also being easily extensible for specific types of problems/errors.

mnot commented 2 years ago

This has been discussed as part of a few other issues, but it's good to separate it out into an explicit one.

I think you're in the ballpark here -- to allow new standard extensions (i.e., those not in the original spec and not explicitly nominated by the type in use), we have two choices:

  1. Find a naming convention that doesn't conflict with existing usage (e.g., a prefix)
  2. Mint a new media type

(1) is hard, because we don't and can't know the extent of existing usage. However, we might be able to create a convention that's extremely unlikely to conflict with existing usage, advertise it widely, and make sure there's a transition period before it's actually used. If we do all that, the risks could be low enough to be acceptable.

(2) is certainly possible, but when we've discussed this in the past, the concern has been that it would further fragment usage.

There is a third option. We can define conventions that individual types opt into, saving them the work of defining the mechanism. For things like problem-pointer (or whatever it becomes), that might be enough.

dret commented 2 years ago

based on the IETF 113 discussion it looks like option (1) with a prefix that isn't too upsetting for JSON users is the best option going forward. it's called "faking forward compatibility when you ignored it initially" and looks good enough to me.

awwright commented 2 years ago

We can define conventions that individual types opt into

How would clients know if a type has opted in or not?

based on the IETF 113 discussion it looks like option (1) with a prefix that isn't too upsetting for JSON users is the best option going forward. it's called "faking forward compatibility when you ignored it initially" and looks good enough to me.

And this enjoys some precedent from some other JSON media types (JSON-LD, JSON Schema). Reserving a prefix seems to have also been deployed in e.g. HTTP cookies (the double underscore prefix) without too much breakage in the wild.

Also, as I sort of hinted, it would be nice to add some stronger language about how to namespace or interpret extension members. Can we guarantee that two "balance" extensions aren't necessarily the same, if the "type" is different? And what do we do if there is no type?

mnot commented 2 years ago

If we choose a prefix, I think it needs to conform to the constraints that I just added to #39 --

When creating extensions, problem type authors should choose their names carefully. To be used in the XML format (see {{xml-syntax}}), they will need to conform to the Name rule in {{Section 2.3 of XML}}. To be used in the HTTP field (see {{field}}), they will need to conform to the Dictionary key syntax defined in {{Section 3.2 of STRUCTURED-FIELDS}}.

That rules out things like $.

Of the two, Dictionary keys are more constrained - their first characters need to be lcalpha or *.

So how about *?

dret commented 2 years ago

On 2022-03-24 22:49, Mark Nottingham wrote:

So how about |*|?

i am fairly open to anything since it's nothing but an opaque thing that we will have to adhere to for future standard members. would it be '" or '-'? i may have a slight preference for the latter for readability.

mnot commented 2 years ago

Either one. *- is a bit weird IMO.

Eg

"*json-pointer": "/foo/bar"

vs

"*-json-pointer": "/foo/bar"

(keep in mind that if #39 doesn't eventuate the possibilities open up quite a bit)

mnot commented 2 years ago

See PR #41

mnot commented 2 years ago

Merging, with the expectation that we'll discuss it on list and advertise this change to make sure we're not clashing with anyone. Even if we don't do the Problem header field now, doing something compatible with SH dictionaries seems like a good idea so that it's future-proof.