postmanlabs / newman

Newman is a command-line collection runner for Postman
https://www.postman.com
Apache License 2.0
6.88k stars 1.16k forks source link

Feature Request: Newman command line switch to only output failed results (i.e. errors) to the output json file (as opposed to outputting passed and failed results). #136

Closed leonvangendt closed 8 years ago

leonvangendt commented 10 years ago

Hi guys, for us Newman would be useful if it could output only failed results instead of both successes and failures.... The advantage is that someone in the team can monitor that file for failures and act on them, rather than have to scan through all the results to find the failures to deal with them. e.g. newman -c apitests.json -b '02 new basket' -e environment.json -o output.json -f

abhijitkane commented 10 years ago

Will look at this for the next release.

llefkowitz commented 10 years ago

It would also be useful to me. I'm doing powershell script to call newman from a task scheduler for the postman collections. Similarly I want all tests to run and get all tests that failed in separate report.

llefkowitz commented 9 years ago

I figured out a way to do this from powershell scripts. Parsing and producing a sum of output, total failures and total successes from each output file. And then sum of all into separate fle, emailing if test failed are > 0. Would be better however to have a flag like -e filename for output errors to file.

shamasis commented 8 years ago

newman now (v3.x) has separate failure and executions array in json reporter

newman run my-collection.json --reporters cli,json

The report will be saved in newman/html-run-report.html

shmup commented 8 years ago

@shamasis removed comment you were probably notified about, because I see that indeed the reporter options are right in the repo README. :)