problem type identifier

[pvdbosch]

In GitLab by @pvdbosch on Nov 14, 2019, 14:02

Proposal to add a standard "code" property to each Problem JSON.

This is inspired on the DigiPolis Antwerp REST guide, but we'd use a more human readable problem code.

• "code" would be a shorter and environment-independent identifier of the type of problem.
• "type" has a environment-dependent dereferenceable absolute URL to a description of the Problem.

possibilities:

"type": "https://b2b-test.ksz-bcss.fgov.be/socialStatus/v1/problems/notFound"
"type": "https://b2b-acpt.ksz-bcss.fgov.be/socialStatus/v1/problems/notFound"
"code": "gCloud:resourceNotFound"
"code": "be.fgov.kszbcss.socialStatus.v1:notFound"
"code": "notFound"

The last option for "code" seemed to be preferred during last WG, i.e. take the last part of the type URL. There is some risk on a new generic gCloud standard problem code overlapping with already existing API-specific ones, but that risk might be small enough to ignore.

[pvdbosch]

In GitLab by @pvdbosch on Nov 14, 2019, 14:37

changed the description

[pvdbosch]

In GitLab by @pvdbosch on Nov 14, 2019, 14:38

changed title from {-environment-independent -}problem code to problem code

[pvdbosch]

up for discussion in a next WG

[pvdbosch]

on a related note: an instance not in URI format doesn't validate (in stoplight) "instance": "d9e35127-e9b1-4201-a211-2b52e52508df"

I've changed the example in common-v1.yaml to the following to make it validate: "instance": "urn:uuid:d9e35127-e9b1-4201-a211-2b52e52508df"

This is a standardized URN format for UUIDs

[pvdbosch]

My proposal:

• keep "type" as unique identifier of the kind of problem, as this is defined as such in RFC and also most commonly used as such
• for API-local types, use URNs instead of URLs, e.g. "urn:problem-type:cbss:ssinReplaced". For gCloud types: continue using URLs; for other non-API-local types: both are allowed.
• add an optional "typeDoc" field which can be dereferenced for more information about the problem type, e.g. "typeDoc": "https://apis.acpt.ksz-bcss.fgov.be/my-api/v1/refData/problems/ssinReplaced"

I think using URNs is better than using relative URIs like zalando recommends because relative URIs should need to be resolved to their full URI to be a unique identifier, and resolving the base URI to which they are relative is context-dependent.

[pvdbosch]

Did some research about what the specs say about URI vs relative URI:

• our openapi-problem schema is more restrictive than the Problem RFC, prohibiting relative URIs
• interpreting a relative URI as being relative to the base URL (defined in OpenAPI) instead of the request's URL deviates from the official standards. For other relative URLs in JSON responses this is also the case, but we recommend using absolute URIs there.

Sources:

• rfc3986:
  • A "URI" is always absolute.
  • A "URI reference" allows both absolute and relative URIs
• in openapi-problem, both "type" and "instance" have format "uri"
• JSON Schema defines format "uri" in OpenAPI as an (absolute) URI (cfr https://tools.ietf.org/html/draft-fge-json-schema-validation-00#section-7.3.6). StopLight enforces this. In later versions of JSON Schema (draft 6 and beyond), the "uri-reference" format is also supported. OpenAPI specs don't mandate support for these format, but formats are extensible.
• rfc7807 Problem Details for JSON allows URI references for type and instance properties. Relative URIs must be resolved relative to the document's base URI (= the URI to which the call is made that returned the problem).

[pvdbosch]

discussed on WG of November 2020, but no conclusion yet. A summary of the discussion follows.

Points raised

1. if something else than absolute URIs are used in "type" (and "instance"), they should according to the Problem JSON spec be interpreted and resolved as relative URIs to get the full problem type identifier. This would lead easily to incorrectly resolved problem types.

For instance, when GET https://something.com/myapi/v1/employers/123 returns a Problem JSON with as "type":

• "/problems/myProblemType" => https://something.com/problems/myProblemType
• "problems/456" => https://something.com/myapi/v1/employers/problems/456

2. Using another field instead of "type" as problem type identifier isn't conform to the Problem JSON RFC.

Consumers MUST use the "type" string as the primary identifier for the problem type;

"type": A URI reference [RFC3986] that identifies the problem type. This specification encourages that, when dereferenced, it provide human-readable documentation for the problem type

Any 3rd party tooling that works with Problem JSON would search for "type".

3. You could use the correct full relative URI as problem type: "/myapi/v1/problems/myProblemType"

However:

• if the API basepath (host or path part) changes for technical reasons, clients don't need only to adapt the REST API's base path, but also any code that interprets problem "type" to do error handling. e.g. https://myhost.com/allMyRestServices/myApi/v1
• the API basepath is different in testing environment (e.g. myhost.acceptance.com), so clients can't reliably program their error handling
  • this is less of a risk for API non-local problem types

4. Using a stable but non-dereferenceable http(s) url as type, is confusing because users might try to dereference it

5. Compliance of definition of identifiers with ICEG URI standard

not investigated much yet

Also see unresolved discussion on ICEG URI standard and REST APIs in https://github.com/belgif/thematic/issues/31

6. Desire to prioritize readability over complexity for problem type identifier (i.e. no long URN structure) - Willem

Option 1

• use "type" only as dereferenceable link to a problem type description, but not as identifier
• introduce a new field "code" that servers as identifier

Example:

{ "type": "https://b2b-acpt.ksz-bcss.fgov.be/socialStatus/v1/refData/problems/ssinReplaced", "code": "ssinReplaced" }

Option 2

• use absolute URNs instead of HTTP URLs as "type" URI
• add another field with a link to the human-readable description
• forbid use of relative URIs in type field

{ "type": "urn:problem-type:ksz-bcss.fgov.be:socialStatus:ssinReplaced", "typeHref": "https://b2b-acpt.ksz-bcss.fgov.be/socialStatus/v1/refData/problems/ssinReplaced" }

This option is similar to this blog but with typeHref added.

TBD:

• the structure of the problem-type URN
• Same for the "instance" field? e.g. "instance": "urn:uuid:d9e35127-e9b1-4201-a211-2b52e52508df"
• transform gcloud problem types to this URN structure as well?

Option 3

• ignore the Problem JSON spec and always interpret the problem types as (non-dereferencable) strings instead of URIs
• a "typeHref" with a link to the human-readable description could be added

{ "type": "/socialStatus/problems/ssinReplaced", "typeHref": "https://b2b-acpt.ksz-bcss.fgov.be/socialStatus/v1/refData/problems/ssinReplaced" }

3a: relative link within base path This option is similar to Zalando REST guide.

3b: relative link includes full base path (without domain name) => increased dependency on technical URIs in application code

Option 4

"type": "https://b2b-acpt.ksz-bcss.fgov.be/socialStatus/v1/refData/problems/ssinReplaced"

type always has absolute URL (env-dependent) clients should parse type to retrieve identifier

[pvdbosch]

My preference would be to maintain compliance with the Problem RFC and restrict our usage further to forbid relative URIs; avoiding complexity to resolve them correctly.

For problem type identifiers either:

• URLs can be used IF they can be kept stable and unique by its author (i.e. not dependent on domain name change, or environment in which API is deployed)
  • best compliant to ICEG URI-standard, but may be too late for the ones which we already defined in gCloud REST guide
• URNs could be used otherwise
  • e.g. for problem types specific to APIs where URLs could be too heavyweight because requires registering types somewhere centrally
  • I'd propose urn:problem-type: with the structure of typeId left up to the author

Other issues:

• add dereferencable link for documentation? "typeHref" - optional or mandatory?
  • or should clients resolve it: /refData/problems/urn:problemType:ssinReplaced
• "instance": should use absolute URI as well?

[pvdbosch]

type should be a URI or relative URI that is env-independent string and stable over time

If it's a relative URL (starting with /) or absolute one, it should be dereferenceable. If it's a URN, not dereferenceable.

type: type: string format: uri => format: uri-reference

not accepted yet ; decision deferred

[pvdbosch]

summarized options in https://github.com/belgif/rest-guide/wiki/problem-type-identifiers , with trade-offs in the table at the end of the page

[pvdbosch]

Results of dedicated meeting on problem type are on wiki:

• chosen solution: https://github.com/belgif/rest-guide/wiki/problem-type-identifiers#chosen-solution
• evaluation criteria: https://github.com/belgif/rest-guide/wiki/problem-type-identifiers#evaluation

[pvdbosch]

WG agreed on chosen solution. We'll document it with rationale for deviation from RFC standard. We'll try to provide this feedback also to the RFC via appropriate channel.

[pvdbosch]

PRs #80 and https://github.com/belgif/openapi-problem/pull/3 ready for review

Not discussed in WG yet: #80 specifies /refData/problemTypes/{problemType} as URL-template for API-specific problem type doc

[pvdbosch]

reviewed together in WG. Merged - will be part of next guide update.