OpenAPI profile

[wetneb]

Citing @osma at https://github.com/NatLibFi/Annif/issues/338

it would make the implementation easier if there was an OpenAPI (Swagger) spec for the API available

I am not familiar with OpenAPI but it does sound like something that would be worth having!

[workergnome]

We've even working on this at the Getty (for our API documentation). It's can be hard to document, since OpenAPI doesn't really have a way to document the "JSON encoded as a property" structure that the OpenRefine API spec uses.

https://gist.github.com/workergnome/afe5b74cff8f1b4fb6490667cf6a4886

[wetneb]

In my (very limited) understanding of OpenAPI, OpenAPI profiles are designed to describe the behaviour of a particular instance of a service, and not a protocol that many services can implement. The OpenAPI profiles I am aware of all include the full URL of the concrete service described.

If that is correct, how can a spec writer provide an OpenAPI profile to an implementer? Do we have any example of specs which provide an OpenAPI profile? Perhaps @osma can give pointers here?

[osma]

That's a valid concern @wetneb. As you say, an OpenAPI spec is generally specific to a particular instance living at a particular server URL.

My understanding of OpenAPI/Swagger is also quite limited and possibly outdated since I've mostly used version 2.0, not the most recent 3.0, but here are some pointers:

The spec file doesn't have to specify an absolute URL path. In OpenAPI 2.0, this is done using schemes, host and basePath; in 3.0, it is instead specified using the servers array which lists one or more base URLs where the service is available. But these don't have to be absolute URLs; they can also be relative to the URL where the spec file is being served on the web. As a concrete example, consider the Annif tool which is implemented using the OpenAPI-spec-driven Connexion toolkit: here, the OpenAPI spec for the Annif REST API (still using 2.0) only states the following:

schemes:
  - http
basePath: /v1

Note that host is omitted; according to the OpenAPI 2.0 spec, "If host is not specified, it is assumed to be the same host where the API documentation is being served." So it is entirely possible to create a spec file that is agnostic about the URL where the service is eventually going to be deployed. That's the case with Annif: you can set up an instance of it on any server and the REST API (including the OpenAPI spec file) will be available on whatever URL it is exposed as.

So I suggest that the OpenAPI profile for the reconciliation API could simply omit the part that states the base URL or use a relative URL.

Note that it is also possible to reuse fragments of OpenAPI specifications, in particular definitions of parameters, data models, responses and operations; see the REUSE.md document for some details.

I don't know of a spec that provides a full OpenAPI profile, but I haven't really looked. But in the generally sensible RESTful API guidelines published by Zalando, in the part that talks about error responses, there is an example of a reusable OpenAPI fragment, published by Zalando, that defines a data model for the "Problem JSON" response (RFC 7807).

[wetneb]

Ok, omitting the host is a good idea.

There is still the problem that the API relies on multiple paths, which are defined in the manifest itself (for instance for the suggest and preview services). Is this convenient for OpenAPI? I suspect it does not support such "dynamic" URLs defined by the response to some query…

We could choose to use sample paths in the OpenAPI profile we provide, but that will mean the OpenAPI spec is more of a suggestion of implementation rather than a specification.

[osma]

What's the reason for having such dynamic URLs? Is it really a necessary feature?

One possible way around this would be to have separate OpenAPI specs for each service/path. Each of them would be self-contained in terms of the spec.

[wetneb]

One nice thing about having dynamic URLs is that you can reuse some parts of existing services. For instance, say someone wants to implement a new Wikidata reconciliation service with a different scoring method: they can reuse the suggest services of another Wikidata reconciliation service in their manifest, so they don't have to implement it themselves. Same for preview and property suggest.

In general, some web frameworks might impose constraints on the shape of URLs, so it might be easier for implementers to choose their own URLs rather than following something imposed by a spec.

One alternative to these dynamic URLs would be to do everything from the same endpoint, using "verbs" passed as GET parameters to distinguish the various commands… I think I have read somewhere that this is bad practice.

I like the idea of having separate OpenAPI specs for each optional part of the API: since not everyone will want to implement all services, it might make the OpenAPI specs more useful.

[thadguidry]

@wetneb And it might make sense that in some cases, those optional parts can actually be OpenAPI extensions x-. Microsoft and others do this alot. There's also some movement from OAI on creating a spec extension registry.

        x-ms-summary: Select List
        x-ms-dynamic-values:
          operationId: GetLists
          value-path: id
          value-title: name

[osma]

Thanks @wetneb for the clarification, I understand the rationale for multiple endpoints, which implies some level of dynamic URLs.

One alternative to these dynamic URLs would be to do everything from the same endpoint, using "verbs" passed as GET parameters to distinguish the various commands… I think I have read somewhere that this is bad practice.

Definitely bad practice, not RESTful at all. Let's not go there...

I think having separate OpenAPI profiles for each service is the best option here. So in terms of OpenAPI at least, the "Reconciliation API" wouldn't be a single API at all, but rather a family of interrelated APIs that play well together but each of them basically independent of the others.

I'm not sure where @thadguidry's comment fits in here. I don't think we need any optional information in that sense in the OpenAPI profile but maybe I just misunderstood.

[wetneb]

To be honest the need to reuse suggest services from other endpoints seems relatively minor, it should not be a blocker. But yes having separate OpenAPI profiles would match the spirit of the existing API pretty well, intuitively.

[thadguidry]

Oh, I was just explaining that if we run into something that cannot be described well with OpenAPI's existing Data Models (Schema) ... we can extend the Schema to fit our needs, as many vendors do, like Microsoft, etc.

[thadguidry]

Since I'm familiar with OpenAPI, I am working on this now... having an OpenAPI spec for all the parts of the Reconciliation API spec. Basing on OpenAPI 3.1 and @wetneb existing draft #172 where I'm also fully incorporating our existing JSON schema, our draft specs descriptions and disambiguation's, as well as external references, etc. So it will be fully complete and fully describing.

I should have this complete by end of June.

[osma]

FWIW, here's another OpenAPI spec (partial, only covers core aspects) from the Annif PR that implements reconciliation API support: https://github.com/NatLibFi/Annif/pull/734/files It might be useful for comparison...

[wetneb]

In the BarCamp, @ostephens suggested that the OpenAPI profile is generated using redocly-cli, which can be used to combine multiple files into a single OpenAPI profile. This would likely help with reusing the existing JSON Schemas as part of the profile: https://redocly.com/docs/resources/multi-file-definitions/