List and document APIs

[cecilia-donnelly]

Document what APIs this module has -- there's an internal division between the external sources and the psm-app proper. What external-facing APIs are there?

For context, this documentation will help us determine which APIs are already here and which will we'll need to create. There are other projects running concurrently with this one (e.g., a reference architecture project), which need to have an overview of what's already included in the PSM. If we can point to code, that's also helpful for them -- the documentation of how to use the APIs doesn't need to be complete right away.

[jasonaowen]

Can you say more about the motivation for this? Is it useful to include endpoints intended for the browser?

We'll also need to talk about authentication at some point to make use of these APIs.

[brainwane]

I'm assuming that it would be better for me to work on a programmatic solution, which would keep up-to-date as we add/change/remove endpoints, rather than spend my time hand-editing the one Markdown file in #100. But if you want me to take a little time to finish up that Markdown file, so we have something merged for now, and then do the foundational work for the longer term, let me know.

I've done some reading and looking today, and found:

I'm looking into the canonical format for publishing the APIs of Java-based projects. I'm not sure yet. Anypoint emits (and thus probably reads) RAML files. I'm not sure yet how the pipeline is going to work -- what bits of the PSM, or what other infrastructure, is going to notice when the list of endpoints has changed and update the RAML file or equivalent.

My assumption is that we'll want to ship two bits of documentation for the API methods:

• within the PSM: web-browseable help files in HTML that include API endpoints & parameters
• on the web, via readthedocs or similar: a page or set of pages including all the API endpoints and parameters, and ideally an in-browser console for developers to try out sample commands

http://raml.org/developers/document-your-api includes a standalone tool that can take a RAML API description and turn it into an HTML page with docs and a console so developers can try sample commands.

Swagger is super popular across lots of different web app stacks and languages for providing autogenerated documentation of APIs, and Jason likes it. I will probably be trying it out using SwaggerHub. An intern of mine wrote up several blog posts while learning to use Swagger a few months ago (example) and I'll start by working off those.

I'll probably play a tiny bit with Blueprint, which Ben mentioned.

[brainwane]

I am putting this work down in favor of a more urgent task; hope the notes above are useful to whoever picks it up!

[brainwane]

Noting the existence of javasphinx (source) and javalink, in case we want to use an approach like the one in this answer.

[brainwane]

I've succeeded in getting very basic javadocs to render for, for instance, ProviderDashboardController. I checked out #190 locally (to avoid a jBPM file that was causing javadoc errors) and ran ./gradlew javadoc in psm-app/, and got a bunch of built HTML files, e.g., psm-app/cms-web/build/docs/javadoc/gov/medicaid/controllers/ProviderDashboardController.html. Sample:

As you see, for viewDrafts, the document mentions:

@RequestMapping(value="/drafts",
                method=GET)

With some trivial addition to the comments for each method, I believe it must be possible to tell javadoc to pull out all the @RequestMapping-annotated endpoints, display them with their associated methods & HTTP verbs, etc.

[brainwane]

The term for what I want Javadoc to do is "custom tag". I haven't been successful in getting the Gradle javadoc task to recognize a custom tag; I believe the next step on that would be to set up a taglet and potentially a doclet.

Next I will take a fresh look at RAML and Swagger and Blueprint now that I understand the space better.

[brainwane]

(Jason also suggested I look into automated Spring tools that emit Swagger definitions.)

[brainwane]

As I see it, we have two choices. (1) we'll futz with Javadoc (and potentially other tools) to get a Javadoc-related toolchain to spit out automated endpoint-and-verb documentation as flat HTML files, or (2) we'll use a toolchain to read our code and emit Swagger or RAML API descriptions, and then render them with the Swagger UI or RAML documentation with its console.

(API Blueprint is out of the running as far as I'm concerned because they don't list a tool to read existing APIs implemented in Java and convert them into the API Blueprint format.)

Regarding the first approach:

I have succeeded in getting Javadoc to produce automatically generated documentation with custom tags: javadoc -d doc-sumana/javadoc/ -sourcepath cms-web/src/main/java/ -subpackages . -tag endpoint:a:"Endpoint: " -tag verb:a:"HTTP Verbs Allowed: ", when run on a file that included the following in a Javadoc comment:

     * @endpoint Endpoint: "/filter"
     * @verb Method: GET

got me

which does display the endpoint and verb!

But I haven't been able to get the custom tags to show up in .rst files produced by javasphinx (javasphinx-apidoc -o doc-sumana/ . -f does work to get it to produce documentation that reflects the Javadocs).

Regarding the second approach:

I've found this overview (it's the intro to a chapter) useful in guiding my thinking. The next thing I want to try is Springfox which "works by examining an application, once, at runtime to infer API semantics" and can then emit a Swagger definition (they aim to eventually support RAML). To do this I need to understand Spring a little better but the approach seems promising. If Springfox doesn't work, I'll work my way through the Java-related RAML tools, such as this RAML generation tool, and I would also want to try out this Swagger code generation Gradle plugin.

[brainwane]

Today (while making #221) I got more familiar with the specific ways we use Spring in our APIs (e.g., org.springframework.web.bind.annotation.RequestMapping) and with what Spring provides and how Springfox is meant to work.

We currently use Spring 3.x. Springfox depends on a component that's only in Spring version 4 and up (https://github.com/springfox/springfox/issues/938, https://github.com/springfox/springfox/issues/921 ). I presume we won't be upgrading to Spring 4.x anytime soon, so next, I'll get one of the Swagger or RAML generation tools running.

[brainwane]

For now we'll use the hand-annotated javadocs approach, as exemplified in #221.

We may want to think about changing how we produce our API endpoints in the future. Right now we use org.springframework.web.bind.annotation.RequestMapping and @RequestMapping is still being used in most of the official guides to making RESTful APIs with Spring, e.g., Building a Hypermedia-Driven RESTful Web Service and Building REST services with Spring. But Creating API Documentation with Restdocs seems to use a different abstraction, MockMvc, and usefully integrates docs with tests.

It's also worth considering JAX-RS. It's part of the Java EE spec, and there are Swagger and RAML tools that consume JAX-RS, but according to https://spring.io/blog/2014/11/23/bootiful-java-ee-support-in-spring-boot-1-2 and https://spring.io/blog/2009/03/08/rest-in-spring-3-mvc/ , the Spring developers aren't as enthusiastic about supporting JAX-RS.

For now I'm putting down this work -- after we upgrade Spring (#219) we'll see what tools are easiest to use, and until then, we'll use the custom Javadoc tags.

[brainwane]

I'm working in this branch and currently fixing these errors.

[cecilia-donnelly]

I think @brainwane has completed this. If we want to move to Swagger or another format we can open another issue.