How to interpret checks that do not fit the template model?

[agstephens]

E.g.:

• first line must be X
• there must be the same number of items in section Y and section Z

Idea: Make the reader for that format undertake those tasks, but include a callback that records them as check results:

• and those results can be included/excluded by general/command-line settings
• e.g. --include-format-checks

It might then be necessary to put all the format checks in a separate config, or have a way of easily listing them - so that you could specify them individually and even decide to switch all/some of them on or off.

Imagine that you could do this:

[settings]
format_checks = group:core-global-attrs group:var-long-names vars-have-standard-name

Where some imply a group of format-checks and others are individual checks.

And/or override with command-line:

checksit check --format-checks=badc-csv:core
checksit check --format-checks=ignore
checksit check --format-checks=all
checksit check --format-checks=default   # e.g. all for that format

How to group and link different checks?

How to group and link them:

• define the format checks individually in JSON - i.e. an inventory of what they are.
• define a separate set of grouped checks - where each group is a list of check IDs along with the parameters that you should send to them
• all this should be definable on the command-line as well as in the config file
• and the actual check functions should remain as generic as possible
• maybe the format check functions can use the regex and func rules defined in the rules sub-package.