docs: reana-client CLI API

tiborsimko commented 4 years ago

Investigate auto-generating mkdocs pages from click/reST documentation.

Use existing library if possible.

Create a helper script if there is no such library.

tiborsimko commented 4 years ago

http://docs.reana.io/reference/reana-client-cli-api

mvidalgarcia commented 4 years ago

I did a little bit of research about this. Here my notes.

Current status

CLI docs are autogenerated thanks to sphinx-click
Source: https://github.com/reanahub/reana-client/blob/master/docs/cliapi.rst
Site: https://reana-client.readthedocs.io/en/latest/cliapi.html

This allows us to have an always-updated CLI API documentation.

Problem

There's no mkdocs-click or similar.
The documentation resides in a different repo now, which complicates making direct references, if any, to the class/method to be documented.

Research

Pachyderm

Pachyderm docs are also based on mkdocs: https://docs.pachyderm.com
They have autogenerated reference: https://docs.pachyderm.com/latest/reference/pachctl/pachctl/
Apparently they generate the docs with a Makefile:
- https://github.com/pachyderm/pachyderm/blob/master/Makefile#L49
- https://github.com/pachyderm/pachyderm/blob/master/Makefile#L313-L317
Which uses a go script: https://github.com/pachyderm/pachyderm/blob/master/src/server/cmd/pachctl-doc/main.go#L24
Which uses cobra library for generating markdown docs: https://github.com/spf13/cobra/blob/master/doc/md_docs.md

So it's no useful for us. Although the way they follow to generate the docs could serve as inspiration (makefile).

mkdocs plugins

There're multiple plugins for MkDocs, some of them related to API documentation building but they more focused on classic pydocs, not specific CLI (click) docs.

...

tiborsimko commented 4 years ago

Our docstrings are providing reasonably complete information already for each command in textual format, for example for the upload command:

$ reana-client upload --help
Usage: reana-client upload [OPTIONS] SOURCES

  Upload files and directories to workspace.

  The `upload` command allows to upload workflow input files and
  directories. The SOURCES argument can be repeated and specifies which
  files and directories are to be uploaded, see examples below. The default
  behaviour is to upload all input files and directories specified in the
  reana.yaml file.

  Examples:

       $ reana-client upload -w myanalysis.42

       $ reana-client upload -w myanalysis.42 code/mycode.py

Options:
  -w, --workflow TEXT      Name or UUID of the workflow. Overrides value of
                           REANA_WORKON environment variable.

  -t, --access-token TEXT  Access token of the current user.
  --help                   Show this message and exit.

So, if there is no ready-to-be-used plugin for our python/click sources yet, we could simply consider our sources as "black box" and generate mkdocs documentation pages solely by means of reusing --help output. As if our sources were written in say Common Lisp :wink:

This way would be faster than writing proper mkdocs-click library or transforming our docstrings to fit some of existing non-click mkdocs plugins for Python.

If we go this way, we could enrich either reana-dev or reana-client to do something like reana-dev --generate-reana-client-mkdocs-pages that will create a ton of markdown pages for each CLI command that we have:

Usage: reana-client [OPTIONS] COMMAND [ARGS]...

  REANA client for interacting with REANA server.

Options:
  -l, --loglevel [DEBUG|INFO|WARNING]
                                  Sets log level
  --help                          Show this message and exit.

Configuration commands:
  ping     Check connection to REANA server.
  version  Show version.

Workflow management commands:
  create  Create a new workflow.
  delete  Delete a workflow.
  diff    Show diff between two workflows.
  list    List all workflows and sessions.

Workflow execution commands:
  logs      Get workflow logs.
  restart   Restart previously run workflow.
  run       Shortcut to create, upload, start a new workflow.
  start     Start previously created workflow.
  status    Get status of a workflow.
  stop      Stop a running workflow.
  validate  Validate workflow specification file.

Workspace interactive commands:
  close  Close an interactive session.
  open   Open an interactive session inside the workspace.

Workspace file management commands:
  download  Download workspace files.
  du        Get workspace disk usage.
  ls        List workspace files.
  mv        Move files within workspace.
  rm        Delete files from workspace.
  upload    Upload files and directories to workspace.

Secret management commands:
  secrets-add     Add secrets from literal string or from file.
  secrets-delete  Delete user secrets by name.
  secrets-list    List user secrets.

Since we don't add commands often, it should not be even necessary to write a full "click parser" at this stage, simply to run a series of CLI invocations to generate these pages one by one. A pseudo code:

for mycommand in upload, du, ls, ...: 
     reana-client mycommand --help > ~r/docs.reana.io/reference/reana-cclient/mycommand.md

We don't even have to massage the output (txt2md) for the first version.

This would give us something to present while we work out more elaborate mkdocs-click integrations. (If there is a need for those at all, actually? I'd see these useful for API docs, but for pure CLI docs, we can really consider our client as a "black box" and only offer what it prints itself, without diving into its Pythonic internals. Might be handy when we'll move to Go one day :wink:)

reanahub / docs.reana.io