Generator for swagger rest api documentation

[stonier]

We've been bundling packages containing all of our rest api's and would like to generate documentation for them alongside with all of the sphinx/doxygen docs.

This bundle lets us do that.

• Parses the package looking for any swagger yaml's
• Dumps the basic swagger html/js files in place
• Template substitutions for name and yaml-json content into the index html
• Presto, done!

[stonier]

There is a repo with example catkin package this can be tested with at here.

[mikepurvis]

How necessary is it to commit all this template stuff to the rosdoc_lite repo? Is there not something we can depend on which would supply that?

FYI @guillaumeautran @ablakey

[stonier]

Example Output

• Web Pages: https://stonier.github.io/catkin_restful/html/robot/index.html
• Rosdoc generated files: https://github.com/stonier/catkin_restful/tree/gh-pages

[stonier]

The templates are a snapshot of the dist folder at the github repo for swagger-ui plus the package.json file to remind us of the version that was nabbed. They also include a couple of modifications in the index.html to remove the general purpose url bar at the top, subsitute the name of the project and insert the json content. Only the index.html has been touched here.

There are quite a few hits for 'packaged swagger ui' on google. They all either bundle something internally, e.g. flasgger, or download it on the fly, e.g. a straight out git clone can get it for you.

Can't assume an online connection, and don't really want to bring in an esotoric dependency on some web packaging framework which will probably assumes an online connection and will be out of date in next to no time, so bundling internally (as ugly as that is) for some stability here is the best solution I can propose. If you've a better idea from guys there more savvy about web stuff than I, I'm all ears.

[mikepurvis]

Understood. Possible alternatives might include:

• Doing a bloom third-party release on the upstream repo, and depending on that.
• Submoduling in the upstream repo, at the most recent tag (since bloom apparently now supports releases with submodules).
• Allowing the user to supply their own template path, perhaps with an envvar (which many would want to anyway).
• Downloading the git tag tarball at build time.

@wjwwood will probably have more to say about this.

[wjwwood]

I don't have any specific feedback. I think downloading is at build time is not a great idea, but I'm not sure embedding it in this package (while practical) is the best idea either. You could do a third-party bloom release, or you could build a deb by hand and ask @tfoote to import it, but either way you'll want to push on upstream to get themselves packaged in a more traditional way, otherwise we'll forever be required to manage that burden of packaging it.

[tfoote]

I agree that packaging it separately makes sense. However until there's traction upstream for a release pipeline packaging it via bloom is probably what I'd recommend. Once there's a pipeline for a debian style package we could import that. But I'd rather not just import a custom debian package on speculation that it might get packaged.

[stonier]

Packaging Swagger-UI via Bloom

Doing a bloom third-party release on the upstream repo, and depending on that.

This would be a step backwards if there's still interest in converting rosdoc_lite to an independent (non-catkin) package since it would then have to do a lookup on the catkin workspace to find it. It also makes it more awkward for distros without packaging support to simply 'drop' rosdoc_lite into the workspace. I think these both outweigh the benefits of having this sit outside rosdoc_lite as a reference source for generating swagger-ui pages, especially since such resources are unlikely to be utilised by anyone else. If someone is going to use swagger-ui, they generally want to customise the web interface to their own needs (see last point here).

Downloading

Downloading the git tag tarball at build time.

Online connection assumed - would prefer to avoid this.

Submodules

Submoduling in the upstream repo, at the most recent tag (since bloom apparently now supports releases with submodules).

Submodules could work (if bloom supports it). This will for sure catch users who just try to do a normal git clone and use the repo. Since we're not intending to contribute back to swagger-ui, git subtree would have less surprises.

Customisation

Allowing the user to supply their own template path, perhaps with an envvar (which many would want to anyway).

This is a good next step. Env Var or rosdoc.yaml variable? Perhaps both.

[stonier]

For reference, relevant discussion about bloom + submodules here

[stonier]

Some updates:

• env variable ROSDOC_SWAGGER_RESOURCES_DIR as a global override
• fallback to a location pointed to by resources_directory in rosdoc.yaml
• further fallback to package local copy of swagger-ui
• some minor pep8 corrections
• added an exception type for plugins to throw

The package local copy of swagger-ui resources is now a submodule and if a user hasn't happened to do the git clone --recursive, then it helps them with a useful error message.

The only test to do from here is to see if it makes it through the bloom release process.

[jack-oquin]

@stonier: I maintain rosdoc_lite as a caretaker, not doing active development. My approach has been rather passive, basically making releases as needed and applying necessary fixes and minor enhancements as requested.

In my estimation, the purpose of rosdoc_lite is for the build farm to generate documentation. The released versions are for local testing of doc updates. I try to discourage people from turning it into a general documentation tool.

I am not sure how this PR fits into all that. Is it an enhancement for the ROS wiki docs? Is it needed for packages you want to release?

[stonier]

I see. If that is indeed the case, then this PR does not fit.

We use rosdoc_lite to build all of our documentation for catkin packages which we then host on internal/external servers. It may not be beautiful, but it is a great tool that does the job of handling documentation in a single way for a variety of documentation generators. Subsequently it's simpler to merge new documentation types (e.g. REST API) as catkin packages than to roll out new documentation generation processes in parallel.

This situation delivers reasonable benefits with low maintenance (we just provide the odd contribution).

We could do something in-house. In this case, labs/companies B, C, 가, 나, 다 are all likely to do something in-house as well as soon as they go beyond supporting a mere handful of packages. I believe @mikepurvis expressed similar sentiments in #66.

I would much rather contribute to the common need of being able to document catkin workspaces rather than spinning up parallelised effort. I also feel that in having an entire suite of build and runtime tools for robotic developers, ROS would sorely miss a tool that enables them to do documentation as well.

However, if rosdoc_lite is not getting active development, and stability for the build farm is a factor in constraining development, then perhaps it is worth considering putting effort into a revamped catkin_doc package which could feasibly eventually replace rosdoc_lite. Not sure I would like to contemplate this with ament/ros2 around the corner, but equally concerned that it may be some time till we manage to move to ament/ros2.

[jack-oquin]

Daniel: My comment was just a personal opinion. I am willing to be guided by others in this.

@mikepurvis, @wjwwood, @dirk-thomas, @tfoote: what do y'all think?

BTW, I will be on vacation for most of the next two weeks, so someone else will need to merge this, if the consensus is to do that before October.

[stonier]

FYI : no rush to get this in, we have the fork building locally.

[tfoote]

With rosdoc_lite not being used in future rosdistros, the age of this, and noetic already out I'm going to close this as feature enhancement and won't fix.