The general idea is good and the approach quite well.
Things to do:
done: rename folder "testsuite.src" to "nistrun.src"
Makefile.am:
add the new testsuite for generation and distribution, see how it is done for the testuite one directory above
new target check-nist, with dependencies nist $(MODULES_ALL) (this will ensure that both the generated testsuite source and the unpacked source files are ready)
either add run_DBNOIX.at and friend and make the complete include conditional depending on the "with_db" variable or - preferably - check the differences and add the appropriate AT_SKIP_IF for the tests to skip
testsuite source files:
change the complete testsuite to not cp anything, but compile from those directories
done: drop the group from the AT_SETUP as we have that in AT_KEYWORDS
done: drop the -x -debug from AT_CHECK - they are already part of $COMPILE
add check of stderr (and, done: stdout) to the compiles, should be completely empty - for some of the early tests add the appropriate -Wno-goto-section-stuff to the COMPILE
This way the general test running should be fine (one test is special as it needs to be killed, just skip it for now), but the important part is the verification (report.pl does some tests, mostly on the generated reports) - I guess that will lead to more AT_CHECK using $GREP/$SED/$AWK, right?
The general idea is good and the approach quite well.
Things to do:
rename folder "testsuite.src" to "nistrun.src"nist $(MODULES_ALL)(this will ensure that both the generated testsuite source and the unpacked source files are ready)AT_SKIP_IFfor the tests to skipcpanything, but compile from those directoriesdrop the group from theAT_SETUPas we have that inAT_KEYWORDSdrop the-x -debugfromAT_CHECK- they are already part of$COMPILEstdout) to the compiles, should be completely empty - for some of the early tests add the appropriate-Wno-goto-section-stuffto theCOMPILEThis way the general test running should be fine (one test is special as it needs to be killed, just skip it for now), but the important part is the verification (report.pl does some tests, mostly on the generated reports) - I guess that will lead to more
AT_CHECKusing$GREP/$SED/$AWK, right?Originally posted by @GitMensch in https://github.com/OCamlPro/gnucobol/pull/83#pullrequestreview-1283694373