For our test runs in CI, we have a test plan that runs multiple test suites in a row. xcodebuild prints out a summary of the test results at the end of each test suite run, but being there are multiple test suits being run, if there's a failure in an early test suite, our devs have to scroll up, and hunt for the failure in the logs. My team and I have been wishing for a way to print out a summary of failed tests at the end of our test runs for quicker triaging.
I noticed today that xcresultparser can print out results to the command line, which is exactly what we need! In our case, we just need the test failures to print, and when passing the --failed-tests-only flag, the output is a little bare. So this pull request is meant to make the printout a little more useful.
My changes aren't complete, so I'd love any feedback on the approach here before I get too far. I'm sure there are particulars about the project that I don't yet understand. Any gotchas in my changes I need to watch out for? Improvements to the approach you can think of?
I'm really loving this tool! Hopefully this is the first of many enhancements I can bring to the project. π
Changes
Adds the name of the test suite to the test detail string
Adds "No test failures" to the test detail string, if outputting to the command line, and --failed-tests-only is passed
Sample Output
Here's a before and after look at the CLI output (I've redacted some of the text here just to protect some private company info):
Summary
Hey there @a7ex π.
For our test runs in CI, we have a test plan that runs multiple test suites in a row.
xcodebuild
prints out a summary of the test results at the end of each test suite run, but being there are multiple test suits being run, if there's a failure in an early test suite, our devs have to scroll up, and hunt for the failure in the logs. My team and I have been wishing for a way to print out a summary of failed tests at the end of our test runs for quicker triaging.I noticed today that
xcresultparser
can print out results to the command line, which is exactly what we need! In our case, we just need the test failures to print, and when passing the--failed-tests-only
flag, the output is a little bare. So this pull request is meant to make the printout a little more useful.My changes aren't complete, so I'd love any feedback on the approach here before I get too far. I'm sure there are particulars about the project that I don't yet understand. Any gotchas in my changes I need to watch out for? Improvements to the approach you can think of?
I'm really loving this tool! Hopefully this is the first of many enhancements I can bring to the project. π
Changes
--failed-tests-only
is passedSample Output
Here's a before and after look at the CLI output (I've redacted some of the text here just to protect some private company info):