API documentation

[MortenHofft]

We increasingly talk about "Powered by GBIF". Meaning that the GBIF provides infrastructure, processing, storage, APIs, UI widgets and websites for the community.

API documentation If that is to succeed we need to build trust. A part of trust is proper documentation, well presented. We have a nice API that is half documented and poorly presented.

Examples of things that is not documented.

• What fields are always present in an occurrence response.
• What fields can I possibly see on a dataset.
• What fields are required when creating a dataset.
• What parameters are available on an occurrence search (we do not document them all).
• What error codes can I expect and what format are the response.

Some of these we could answer with out current approach, others would be more difficult.

What tools? I do not know much about documentation formats, but OpenAPI Specification (formerly Swagger Specification) seems like a fine candidate to me. We could manage it by handcrafting it or we could generate it from annotations. I guess both would do. I do not mind doing it by hand, but I would need to ask the rest of you questions.

What do the rest of you think? Another format? Keep it as is? Annotate the code? Something else?

[timrobertson100]

What do the rest of you think?

In no particular order:

• It's great that someone is looking to improve this, as I agree. Thank you
• I tend to suspect keeping documentation out of code (i.e. handcrafting) is better because:
  • it does not tie us to the code release cycle (quick fixes can be applied in minutes)
  • code is scattered, and you have to find the project learn to build etc
  • non-developers can contribute easier
  • we code in different languages and surely will hit issues integrating different builds
• Following a standard like OpenAPI seems sensible to me
  • Especially if it helps with e.g. generating a stub for GraphQL or some other client
  • documenting expectations on model formats seems good ("what is the schema for object X" type questions)
  • stops us inventing formats (handcrafted OpenAPI specification yaml seems appropriate to me)

[mdoering]

Yes please! I can tell that we picked RAML to document the CoL API, but I sometimes wonder if that was a wise decision as its super complex, lacks good tooling and our API is still evolving considerably. On the other hand if you know how to use it its very powerful and you can do inheritance, "traits" and a lot more to avoid tedious repetitions e.g. for reoccurring CRUD methods or paging. It does also do serious validation of examples that can be embedded

[mdoering]

https://github.com/Sp2000/colplus/blob/master/docs/api/api.raml

[MortenHofft]

The stub I created to test the openAPI specification is rendered using ReDox and visible at http://labs.gbif.org:7018/

@asturcon did a nice example for the vocabulary API using Spring Rest Docs

[MattBlissett]

I have moved this from a different repository. Much of it is done, but I think it should be considered when we advertise the tech-docs site for internal review — have we met Morten's requirements?

https://tech-docs.gbif-uat.org/en/openapi/ (wonder if the URL should end /api/ rather than /openapi/?)

[MattBlissett]

Morten:

Examples of things that is not documented.

• What fields are always present in an occurrence response.

This is still not well documented with the new OpenAPI documentation. I'm not sure if it's still correct, but there are @Nullable annotations on things like installationKey, lastCrawled, crawlId, basisOfRecord.

• What fields can I possibly see on a dataset.

This is documented, by default as an example response with nonsense values, and by clicking "Schema" for the definitions.

Occurrence objects can have all the verbatim (uninterpreted) properties. These are not yet properly documented.

• What fields are required when creating a dataset.

The "Schema" view marks the required fields with a red *.

• What parameters are available on an occurrence search (we do not document them all).

https://tech-docs.gbif-uat.org/en/openapi/v1/occurrence#/Searching%20occurrences/searchOccurrence

• What error codes can I expect and what format are the response.

Included, but it will be worth checking these before release.

---

Tim:

I tend to suspect keeping documentation out of code (i.e. handcrafting) is better because:

• it does not tie us to the code release cycle (quick fixes can be applied in minutes)
• code is scattered, and you have to find the project learn to build etc
• non-developers can contribute easier
• we code in different languages and surely will hit issues integrating different builds

I put annotations on the code, e.g. https://github.com/gbif/occurrence/blob/dev/occurrence-ws/src/main/java/org/gbif/occurrence/ws/resources/OccurrenceSearchResource.java#L217, since

• I think the developers are far more likely to update this when they make changes to the webservices. Documentation in other places is easily forgotten
• Non-developers will lack knowledge or confidence to update the API documentation anyway
• Handcrafting OpenAPI seemed overwhelming. It would be easy to miss things, or to have inconsistencies. With annotated Java, we can even add test cases later if we want to, to check the documentation is complete.