search

ASSERT-KTH / spork

AST-based structured merge tool for Java, fully Git compatible https://doi.org/10.1109/TSE.2022.3143766
MIT License
49 stars 6 forks source link

Make all tests silent when they succeed #510

Closed wetneb closed 5 months ago

wetneb commented 5 months ago

The test suite currently outputs a lot of things on the standard output. It means longer build logs and makes it difficult to identify quickly which tests are failing and why.

It is generally a good practice to make sure that passing tests do not output anything on stdout and stderr. This adjusts the tests for that. The merge outputs are still shown when the test fails.

codecov-commenter commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.43%. Comparing base (ba76361) to head (8b0491e). Report is 5 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #510 +/- ## ============================================ - Coverage 82.70% 82.43% -0.28% Complexity 363 363 ============================================ Files 43 43 Lines 1769 1719 -50 Branches 303 303 ============================================ - Hits 1463 1417 -46 + Misses 180 176 -4 Partials 126 126 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

slarse commented 5 months ago

Totally agree on all your points. I do however feel like the test harness should handle these kinds of things. That's what I'm used to from other technologies and I find it works well.

It doesn't seem like Maven (Surefire) nor JUnit has any such options, but it's pretty easy to hook into JUnit to do it. See #512, let me know what you think. Does it address your issues?

The improved formatting from this PR could be combined with that as well.

wetneb commented 5 months ago

Makes sense, it works well :)