Optimize passing tests computation

STAMP-project / test-runner

test driver to run JUnit tests isolated in a new JVM

GNU General Public License v3.0

16 stars 15 forks source link

Optimize passing tests computation #114

Closed andre15silva closed 3 years ago

andre15silva commented 3 years ago

Test results are now stored as sets so operations on them, such as the passing tests computation, are closer to constant.
Fix incoherence between JUnit5 and JUnit4 listeners, where JUnit4 wouldn't check istTest().

Note that JUnit5 would allow for the test results to be stored immediately as passing, failing, etc, instead of having to do such computation, but the JUnit4 listener doesn't provide this information in testFinished, and testFinished is run after testFailure not allowing for such computation to be done while running the tests.

~~To be rebased and merged after #113~~

coveralls commented 3 years ago

Pull Request Test Coverage Report for Build 1104322819

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

16 of 16 (100.0%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 45.237%

Totals
Change from base Build 1101644940:	-0.1%
Covered Lines:	1078
Relevant Lines:	2383

💛 - Coveralls

andre15silva commented 3 years ago

Could you give me a case where the description.isTest() would give false?

Yes, as far as I understood it is false when the description represents a TestSuite (see http://junit.sourceforge.net/junit3.8.1/javadoc/junit/framework/TestSuite.html). From the javadoc of Description:

A Description describes a test which is to be run or has been run. Descriptions can be atomic (a single test) or compound (containing children tests).

But now that I think of it, the listener already does that filtering for us, since any of those methods are only called for atomic tests. I'll revert those ifs.