search

exercism / julia-test-runner

GNU Affero General Public License v3.0
2 stars 8 forks source link

Collapse testsets #77

Closed cmcaine closed 2 years ago

cmcaine commented 2 years ago

Fixes #53.

Please don't squash these commits: there are quite a few changes and the atomic commits make it easier to understand what's going on + the changes to the fixtures between 154e5a9 and c3b5396 make it easy to see what the effect of this PR is without having to run the tests yourself.

Changes:

a7b986f (Colin Caine, 53 seconds ago) Report test_code for passing tests

c3b5396 (Colin Caine, 10 minutes ago) Report fewer results

Multiple passing results are collapsed into a single report. At most 5 failing results are reported per testset.

154e5a9 (Colin Caine, 5 minutes ago) Add fixtures to test collapsing behaviour

2c7d29f (Colin Caine, 6 minutes ago) Update error messages in fixtures

cmcaine commented 2 years ago

Thanks for the review! I couldn't think of a nice way to show a warning and I wasn't convinced it would be very helpful, so I think we should merge this without adding one. Can always do another PR if it seems like a good idea.

I'll rebase and merge this later today or tomorrow if you've no objections :)