unittest improvements

[FedericoCeratto]

A couple of ideas that could be implemented in unittest or dedicated tools/libs:

• [ ] Print a test summary at the end (# of passed, failed, skipped tests, ... )
• [ ] JSON outputs
• [x] JUnit #2088 outputs
• [x] Syntax or examples to perform once-for-all setup and teardown
• [ ] Nested tests where a parent tests will fail if any children fails
• [ ] "expected failure" test flag that reverses the pass/fail output status
• [ ] Additional colorscheme for light (white) backgrounds
• [x] Filter tests to execute by name
• [x] Filter tests to execute by glob/regexp on name
• [x] Filter tests to execute by suite name
• [x] Filter tests to execute by glob/regexp on suite name
• [ ] Flag to stop on first failure
• [ ] Modular fixtures e. g. http://doc.pytest.org/en/latest/fixture.html#fixture
• [ ] Track which tests cover which lines of code and re-run tests selectively based on code changes
• [ ] Distributed compilation and testing
• [ ] Allow running multiple test files and merging the outputs in one report and junit file

Test execution (it could be run though Nimble):

• Discovering tests by filename (defaulting to test.nim test.nim test/.nim tests/.nim )
• Run build commands before testing
• Run tests upon file change
• Flag to re-run only failed tests

[FedericoCeratto]

Some improvements in #4639

[FedericoCeratto]

A new tool for test discovery and run: https://github.com/FedericoCeratto/nim-testrunner

[FedericoCeratto]

JUnit implemented in #5556

[narimiran]

Any recent news on this one?