Uncertainty created by relative URIs used as IDs for problem "type" field

[benbucksch]

You are specifying problem "type" IDs as URIs. That's a good idea, it allows for scoping and teams to assign their own error IDs.

However, allowing relative URIs creates significant real world problems:

• Most of the time, parsing of the errors will be adhoc. The chances that the client does not resolve the relative URLs into absolute URLs is high. This creates potential for conflict and/or multiple IDs.
• APIs may be available on different endpoints (URLs). Depending on the endpoint, that would create different IDs. That is not desirable. For a realistic example, a dev uses a small npm lib to parse Problem errors, and the lib correctly resolves the relative URI in "type" into absolute URIs. The dev treats errors based on these these types with absolute URIs. He and QA are testing against a dev or staging server and they do not know or realize that the production server will have a different API endpoint URL. So, it works in testing, but fails in production. This is a likely scenario. Reliable testing would be difficult. Using relative URI for "type" is setting up for such a problem.

I would recommend to either 1) mandate absolute URIs or alternatively to 2) define the ID as free-form generic string. In the latter case, relative URLs would not be resolved, but everybody would use them as strings - which is what likely happens in reality anyways.

[darrelmiller]

@benbucksch I don't think you will get any disagreement from the folks lurking here that type should be an absolute URI, however there is a significant amount of community feedback that requiring an absolute URI would impact adoption https://github.com/ietf-wg-httpapi/rfc7807bis/issues/11

It would also be a breaking change to RFC 7807 which would fragment the ecosystem and require a new media type registration.

[benbucksch]

@darrelmiller Thanks for the link to #11 and the Twitter thread quoted there. I can surely see why absolute URIs are unpopular. That is esp. true when there is an expectation that they are resolvable or might be resolvable in the future. It immediately triggers an internal discussion about which hostname to use, which format, which URL is the most stable to host long-term etc.pp., and that internal discussion causes considerable delay and internal friction. So, I can see why pure strings for "type" are more popular and make adoption of the spec far easier.

This issue is not contradicting #11. Rather, this issue is about relative URIs specifically. We should avoid the use of relative URIs in this particular use case. While they seem like an elegant solution to allow simple strings, they are actually harmful here in specific situation. To put it into a punchline: Stable and relative doesn't match well.

Common practice is that API endpoint URLs are configurable, and to have different endpoints for testing and production environments, so using relative URIs pretty much guarantees that the "type" IDs are not stable, and even change between testing and production environment, which leads to serious problems. The worst kind of bugs are which the customer sees, but the dev cannot reproduce in his dev environment, and the relative URIs (if they are processed in a spec-conforming way) are provoking exactly that scenario.

[benbucksch]

I can see the following possible solutions:

1. Demand that "type" is always an absolute URI.
   • Based on #11, I can see why that is a non-starter, so let's forget about that option and move on to other solutions.
2. Define "type" to be an arbitrary string.
   • This allows for absolute URIs, but also allows more relaxed assignment of error IDs.
   • It is easier to parse, because it removes the requirement to resolve relative URIs to absolute URIs in order to get the actual type.
   • It is more stable than relative URIs, because relative URIs are pretty much guaranteed to change.
3. Remove the recommendation that the type ID is a resolvable URI with error documentation, and instead add a second "documentation" field with a link. This removes the coupling between the ID, which needs stability, and hosting of webpages. Very few orgs in this world are capable of guaranting their web URLs to be stable over the next 20 years. Many don't even know what happens in 2 months.

I would recommend options 2 and 3 above. It's easy to use, it's backwards-compatible, and relaxes the discussions of web hosting before assigning error IDs.

It also allows devs to easily add new error IDs without worrying about creating a web page for it, translating it, etc. This is turn avoids that devs lump multiple errors into the same type ID, which defeats the purpose.

[asbjornu]

Hi, @benbucksch. On e-mail to the list, you wrote:

As mentioned in my feedback, I would recommend to not couple resolvable description webpage URLs and stable error type IDs. If even IANA has troubles providing this right now, can you imagine the discussions in companies and within projects?

As RFC 7807 is from 2016, that discussion has been ongoing for more than 6 years and I haven't seen any convincing arguments during that time to change how type works.

Just publishing anything on the website is often a major process for many companies. And then guaranteeing stability on top of that…

Guaranteeing stability of type URIs is no more difficult than guaranteeing stability of URIs used in an API. As most HTTP APIs today are not hypermedia-driven, clients hard-code URIs and thus require them to be stable.

Even if the webpage is optional, there'll be the discussion that we might in the future make a page, so better choose wisely now... But which hostname? Which URL form?

My recommendation is to use the same base URI as in the API for type URIs. For example:

https://api.example.com/problems/out-of-funds

Since problems now is a programmable resource within the API, it's easy to redirect from the above URI to:

https://www.example.com/docs/reference/problem-types

Whenever the URI of the documentation changes, it's just to change the coded redirection within the problems resource. Simple, no?

[benbucksch]

As RFC 7807 is from 2016

And neither PayPal nor Stripe nor any other API that I ran into uses it.

@asbjornu The problems mentioned in the initial comment are real. Let's please try to solve them. I've made a few suggestions.

[awwright]

Demand that "type" is always an absolute URI

If this is a concern in your implementation, then you can use (full) URIs, I'm not sure how mandating them for everyone resolves an interoperability concern.

(Small nit: "absolute URI" is an ambiguous term, it can refer to either the fragmentless URI like what HTTP servers receive, or what we would normally call just a URI as opposed to a URI reference.)

Define "type" to be an arbitrary string

This would break implementations that use URI references: You should be able to use URIs and URI references interchangeably. It would also create a namespace collision, since "type" is supposed to be globally unique.

Remove the recommendation that the type ID is a resolvable URI with error documentation, and instead add a second "documentation" field with a link

I'm not sure this fixes the problem? Though it is true that the URI identifying the error type, and a document about the error type, are not necessarily the same resource.

[benbucksch]

in your implementation

I'm not talking about myself, but about typical professional setups. Almost all pro setups will have a staging environment for testing before going into production, and they are all affected.

[awwright]

@benbucksch My point applies regardless of who "you" is; if a URI reference in your staging environment isn't the behavior you want, then use a URI instead. I don't think it makes sense to say "URI references don't make sense in some situations therefore they should be prohibited." What about the situations where they do make sense?

Also note that multiple different URIs are allowed to identify the same resource; and likewise, two URIs that are different don't necessarily identify different resources. Resolving multiple different URIs that identify the same type is by no means wrong.

[mnot]

@benbucksch the specification already warns about the issues involved when relative URIs are used. Given the constraints that others have mentioned, I don't think we can do much more.

If you can suggest ways to improve those warnings, that'd be helpful.

[darrelmiller]

Closing this issue. @benbucksch if you have suggestions on how to improve the warning, feel free to re-open.

[benbucksch]

The issue I reported is a real-world problem. Whatever the solution, there needs to be some kind of solution that addresses the issue.

If none of the suggestions are acceptable, and nobody has a better idea, then the safest option is to deprecate relative URLs in the ID field, for the harm that they can cause.

" @deprecated Simple string and relative URLs in the type field are deprecated. Some existing software might still use it, but such software should be changed to use absolute URIs instead. If you use relative URLs in the type field, and the client correctly resolves it into an absolute URL, and then processes the type further, and you have a testing and production setup on different hostnames or different URLs, then the client may behave differently in production than in testing. Some error handling may work in your testing environment, but will not work in production. To avoid such surprises, relative URLs are discouraged and should no longer be used. A simple solution is to use your company's domain name, and your app name, and construct a non-resolvable URI like "https://error.example.com/myapp/file-not-found" and use this as type. "

@darrelmiller I do not have the ability/rights to re-open.

[mnot]

That's not a suggestion for improved wording, that's recycling a proposal that failed to get consensus.

[benbucksch]

If you have any other suggestion how to fix the original issue, I'd be happy to hear it.

[awwright]

@benbucksch Both of the identified "problems" are useful features that people might want to rely on, you're just pointing out that some people might not use them correctly, but I'm not sure that implies "deprecating" is a solution.

If the feature is too confusing for you then don't use it; you can guarantee to your clients you'll always use a URI.

Have you personally seen these "real world" problems?

[benbucksch]

If the feature is too confusing for you

It is not. As I stated above already in response to you, this issue is not about me personally.

You're questioning the scope of the problem. The scenario I outlined is a problem for everybody who

• uses relative URLs as type ID (100% of people concerned by relative URIs discussed here)
• correctly resolves them on the client (100% of people who use the spec as specced)
• has specific error handling for some specific errors, which he recognizes based on the type as absolute URI, as the spec intended (again, all the people who do error handling and use the spec correctly as intended), and
• has a separate qa/testing/staging and production server setup (i.e. everybody with a proper testing setup), (or the same API is available on multiple endpoints/server URLs for any other reason).

[awwright]

@benbucksch Well, again, my point applies regardless of who "you" is. But also you said "real world", as if this is a problem you've seen previously.

The scenario I outlined is a problem for everybody who

The scenario you outlined is one where the application should use a constant, full URI and not a URI reference. That solves the problem without removing a feature that's useful for other people.

What you have to explain is why you (in your hypothetical) are using a URI reference instead of a full URI, then why removing URI reference resolution is the solution to this problem instead of just changing to a full URI in the Problem document.

The fact that lots of file formats use relative URI references without these problems also suggests to me this is a hypothetical problem and not a real one; you have to also explain what's unique about this specification that doesn't apply to anywhere else URI references are deployed.

[dret]

On 2022-12-22 09:35, Austin Wright wrote:

What you have to explain is why you (in your hypothetical) are using a URI reference instead of a full URI, then why removing URI reference resolution is the solution to this problem instead of just changing to a full URI in the Problem document.

what seems to be repeatedly getting lost in this never-ending discussion is that this is a revision of an existing spec. breaking changes are not on the table. deprecating existing features is not an option. the revision has added cautionary text and that's all that possibly can be done.

happy holidays everybody!

-- Erik Wilde | @.*** | | https://youtube.com/ErikWilde |