Make target to check doc examples against actual output

[hickeng]

Story As a user I expect documentation to be precise and correct for the version it relates to As a VIC developer I want automation to support me in making necessary documentation changes when altering user facing code

Details We have had several instances of documentation not being updated in PRs that change user facing code, and others where the docs have incorrect data (e.g. #4817). The latter is harder to address, but ensuring that examples are correct is a simple matching problem. This story should add:

• markdown convention surrounding any CLI examples
  • command to run
  • filter for output
• make target that confirms that example output matches that from actual invocation of the command

[stuclem]

@hickeng this is a good idea that you have raised several times before. However, I see a couple of potential issues:

• The doc source files are in the vic-product repo rather than in vic. Is this a problem?
• In most cases, we tag commands, output, and code snippets with HTML rather than with Markdown. This is because Markdown does not allow us to format commands, etc., in the way that we want them to appear in the final doc. In particular, Markdown does not allow you to nest tags inside its code tags. So, in order to adhere to the industry standard of italicizing variables, one has to use HTML tagging. Line wrapping is also problematic.

[hickeng]

@stuclem I was actually referring specifically to the contents of the doc folder in this repo and the packaged READMEs - but if we do do this, then we should endeavour to have the target be of use in the formal docs as well. Whether using markdown or HTML the requirement is much the same: a. be able to identify a command to check, and any pre-requisite state b. minor transforms to apply to the command (e.g. add -linux to vic-machine as we tend to drop the platform specific portions) c. run the command d. minor transforms on the output, for example stripping timestamps e. compare with current example output

The idea here isn't to generate the example output directly, but to break our build if the example output is out of date to force the developer to update it.

[stuclem]

OK, thanks for the info @hickeng. I won't create a corresponding doc issue in vic-product until such time as we identify clearly what changes might be needed in the formal doc. Leaving the kind/user-doc label in place on this one.