DannyDainton / newman-reporter-htmlextra

A HTML reporter for Postman's Command Line Runner, Newman. Includes Non Aggregated Runs broken down by Iterations, Skipped Tests, Console Logs and the handlebars helpers module for better custom templates.
Apache License 2.0
357 stars 139 forks source link

Failed Request count is always zero - even if an Request had assertion errors #246

Closed svarshney2311 closed 4 years ago

svarshney2311 commented 4 years ago

When assertions within an Request fail, I expect the Request fail count to increase. Instead, it remains at 0. This way, I cannot count successful Requests, only "run" Requests.

image

image

DannyDainton commented 4 years ago

That number relates to the number of assertions that have failed - As you can see from your image, the number on the failed assertions, matches the total in the tab.

If you were to open that tab, you wound see that a single assertion had failed with a test.

svarshney2311 commented 4 years ago

But the requirement is If any assertion fails, Test Requests should be failed as assertion link to the Test Request. Here in the example, even the assertion is failed but still Test Requests showing as passed,that looks something odd and this issue had raised by Client also.

DannyDainton commented 4 years ago

Who's requirement? 🤔

I'm telling you, from the person who created the reporter and the logic behind why things show up in different parts of the app - Also from someone who knows a few things about the way Postman works...it's doing what I expect it too 😁

How many assertions failed? It looks like 1 and 1 was placed in the label at the top.

A test failing doesn't equal a request failing, a request can fail without it even having a test within it.

A failed request could be due to an invalid URL being used, an invalid header etc. If has nothing to do with a failed assertion, the assertion is made after the request has been sent.

github-actions[bot] commented 2 years ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.