search

gerryjackson / forth2012-test-suite

Test programs for Forth 2012 and ANS Forth
72 stars 15 forks source link

Error Reporting does not distinguish test not run and no errors found #1

Closed steverpalmer closed 8 years ago

steverpalmer commented 9 years ago

After running the tests, a summary report is generated which lists the numbers of faults found for each test suite. If no faults are found, then the summary line displays the test suite name and a "-". This is fine, but is also used to present the results for a test suite that has not been run. From a configuration management perspective, it would be better to display something different for those tests that have not been run. Perhaps "n/a"?

steverpalmer commented 9 years ago

For example, I commented out several test suites in the runtests.fth file, and then ran the tests and got the following output...

---------------------------
        Error Report
Word Set             Errors
---------------------------
Core                    -
Core extension          -
Double number           -
Exception               -
Facility                -
File-access             -
Locals                  -
Memory-allocation       1
Programming-tools       -
Search-order            -
String                  -
---------------------------
Total                   1
---------------------------

There is no indication that the Double number, Locals, Programming-tools and String tests where not run.

gerryjackson commented 9 years ago

I think that is an excellent idea that we should implement. Are you volunteering to do it?