Generate REST documentation from OpenAPI specification

[mboskamp]

User Story (Required on creation)

As a user, I want to find up-to-date information about the REST API on a dedicated documentation page. As a developer, I want to generate the REST API documentation from the existing OpenAPI specification to reduce the manual maintenance of two sources.

Functional Requirements (Required before implementation)

• The REST documentation website is available at https://docs.camunda.org/rest/camunda-bpm-platform/7.19-SNAPSHOT/
• The documentation is generated from the OpenAPI spec
• The documentation is updated and released with every patch/minor/alpha release
• CMMN endpoints won't be covered initially; we can add them later if we have evidence that users need them

Technical Requirements (Required before implementation)

• The documentation can be generated by triggering a CI job
• The release pipeline has a new step to generate and publish the generated documentation
• Redirects are in place to map old REST documentation pages to the new page

Breakdown

• [x] #2847
• [x] backport to 7.18 camunda/camunda-bpm-platform-maintenance/pull/871
• [x] backport to 7.17 camunda/camunda-bpm-platform-maintenance/pull/872
• [x] backport to 7.16 camunda/camunda-bpm-platform-maintenance/pull/873
• [x] backport to 7.15 camunda/camunda-bpm-platform-maintenance/pull/874
• [x] camunda/jenkins-job-dsl-seed-jobs/pull/374
• [x] camunda/cambpm-jenkins-shared-library/pull/96
• [x] #2864
• [x] camunda/jenkins-job-dsl-seed-jobs#376
• [x] camunda/camunda-docs-static/pull/304
• [x] Adjust REST API links in docs: camunda/camunda-docs-manual/pull/1344
• [x] Adjust release guides

[tasso94]

Hi @mboskamp,

I submitted the following PRs for review:

• https://github.com/camunda/camunda-bpm-platform/pull/2847
• https://github.com/camunda/jenkins-job-dsl-seed-jobs/pull/374
• https://github.com/camunda/cambpm-jenkins-shared-library/pull/96

You can find the CI results either directly on the PR or in my comments.

Best, Tassilo

[mboskamp]

@tasso94, The following PRs are ready for (re-)review:

• https://github.com/camunda/camunda-docs-manual/pull/1344
• https://github.com/camunda/camunda-docs-static/pull/304 (should be merged last)

[mboskamp]

Some things that came to my mind while working on the new docs:

• For some REST docs pages we grouped multiple endpoint in one markdown file (example). In the new docs, it would be nice to have those endpoints close together (i.e. after each other in the list) so they can be easily found. We should think about ordering the docs.
• Should we maybe announce the new docs in a blog post and in the forum? I could imagine that users would be confused if we did not communicate the change.

[tasso94]

Hi @mboskamp,

I added some review hints on your PRs.

Since this is a bigger impactful change, can we publish redirects + documentation changes on stage before we merge the PRs to test it a little bit manually? You can involve in that too.

Regarding your comments above:

• I'm surprised this is not the case; can we change the order easily?
• I think this is a PM and an EM question; I can clarify this with them if that's fine for you.

Best, Tassilo

[mboskamp]

@tasso94, I implemented the review hints and left a response in two cases because there are open questions.

Do you think it would be too high of a risk to merge the PRs to test them on stage? What other option do we have to publish the redirects + documentation on stage?

Regarding the ordering of the endpoints: According to one of the developers the endpoints are grouped by tags (obviously). The tag order is the one defined in the tags object. Inside a tag the endpoints are ordered as they appear in the JSON. I believe we don't apply any special order when generating the JSON so they appear in the order the files are structured in the file system.

[tasso94]

@mboskamp, I thought about it more, and the minimally invasive approach would be:

1. Only merge the changes for stage/.htaccess with master
2. Run https://ci.cambpm.camunda.cloud/view/Docs/job/docs/job/camunda-docs-static-config/
3. Merge the changes for camunda-docs-manual with master
4. Job to stage docs is automatically triggered
5. Try out if everything works as expected on stage
6. Apply changes to redirects and/or merge the changes for live/.htaccess with master as well
7. Run https://ci.cambpm.camunda.cloud/view/Docs/job/docs/job/camunda-docs-static-config/
8. Make the develop docs public https://ci.cambpm.camunda.cloud/view/Docs/job/docs/job/camunda-docs-release%20(manual-develop)/

The worst that can happen if we merge everything without some stage testing is that the docs experience downtime. That has happened to me at least once in the past due to an erroneous rewrite rule. That might impact internal departments and external users (might create some load in support and the forum). Also, our Google ranking could be affected if we don't revert immediately.

Should we create a follow-up ticket to improve the order of the tags?

[mboskamp]

@tasso94, I scheduled the testing for Monday 21st. Let's use the list from your previous comment to merge the docs-manual changes and redirects to stage so we can test it a bit.

For the ordering of the endpoints in the docs I created a follow-up issue: #2958

[mboskamp]

The testing is done and the changes are reverted for now. We will merge everything with the next alpha release (7.19.0-alpha2) on December 12th.