Make sure most (ideally all) ODK programs are tested to run

[gouttegd]

We’ve just had a case (as part of #1076) where one of the programs provided by the ODK (j2) was broken to the point that it could not even run. That program is not used in any of the standard ODK workflows, so it is not tested as part of the test suite and the issue was only caught after @matentzn did some extra tests on one of the ontologies he is involved with.

While it may not be realistically feasible to include in the test suite some representative examples of the diversity of custom workflows that are used out there (and this is why the kind of extra testing we do on real-world ontologies will always be necessary), we should at least try to ensure that, if a program is provided in the ODK image, it can at least run without crashing or erroring out immediately.

There is already a basic infrastructure in place to test running some of the most important ODK programs:

test_odklite_programs:
    @./tests/test-program.sh ROBOT robot --version
    @./tests/test-program.sh DOSDP-TOOLS dosdp-tools -v
    @./tests/test-program.sh OWLTOOLS owltools --version
    @./tests/test-program.sh AMMONITE sh amm --help
    @./tests/test-program.sh ODK odk.py

test_odkfull_programs: test_odklite_programs
    @./tests/test-program.sh KONCLUDE Konclude -h
    @./tests/test-program.sh SOUFFLE souffle --version
    @./tests/test-program.sh JENA jena
    @./tests/test-program.sh SPARQL sparql --version

This does not guarantee that the programs are fully functional, but it can at least catch basic issues like:

• for compiled, binary programs: a missing dynamic library (this happened at least once over the past three years, with the Konclude reasoner – this was actually the incident that motivated the addition of the tests shown above);
• for Python programs: a missing Python module, or an incompatibility between some modules or between a module and the version of Python used in the ODK (what happened for j2);
• for Java programs: an incomplete Jar bundle archive, or an incompatibility with the JVM provided by the ODK;
• etc.

We should generalise the tests above for most of, if not all, the programs provided by the ODK, regardless of whether they are used in the standard workflows (especially if they are not used in the standard workflows).

[gouttegd]

Two things to do here:

First, for programs that are already part of the ODK: we need an inventory of them, so that we can add them to the existing tests.

Second, I’d like for the ODK project to adopt the following policy: whenever someone creates a PR that (among other things) adds a new program to the ODK, the PR must include a basic test that the program can at least run. (With the exception of programs that are provided by the Ubuntu distribution and that are installed by apt-get: for those, it’s fine to trust the Debian/Ubuntu package developers.)