postmanlabs / newman

Newman is a command-line collection runner for Postman
https://www.postman.com
Apache License 2.0
6.88k stars 1.16k forks source link

is there any way to display pass percentage in terminal/html report ? #1436

Closed jayanthktn closed 6 years ago

jayanthktn commented 6 years ago

Postman for Windows Version 5.3.2 Win 7 / x64 Newman 3.9.3

My request is to display the pass percentage in the console log - Much easier for management people. Additionally adding percentage of passed Test cases . 79 passed out of 93 means = 84.94%..

Something like that.

kunagpal commented 6 years ago

While this isn't possible in the default reporter, setting percentage reports up using a custom reporter is quite straightforward, as can be seen below.

  1. You can start off by checking out our docs for custom Newman reporters here: https://www.getpostman.com/docs/postman/collection_runs/command_line_integration_with_newman#custom-reporters

  2. Next, you can build on top of the existing HTML reporter to add your own helper for percentage calculations. This can be done by including the following snippet in your custom reporter code:

    handlebars.registerHelper('percent', function (passed, failed) {
    return (passed * 100 / (passed + failed)).toFixed(2);
    });
  3. Now, you can use compute percent values anywhere in your report by including the following statement in the handlebars template:

    {{percent cumulativeTests.passed cumulativeTests.failed}}
  4. Note that the javascript code for your custom reporter will have to reference the path to your modified handlebars template.

Feel free to revert if you have any questions 😄

jayanthktn commented 6 years ago

I am getting the following error .

\helper-missing.js:19 throw new _exception2['default']('Missing helper: "' + arguments[arguments.length - 1].name + '"'); ^ Error: Missing helper: "percent"

Here's my main code,

***

Newman Report

{{#with summary}}
Collection
{{collection.name}}
{{#if collection.description}}
Description
{{collection.description}} 
{{/if}} {{/with}}
Time
{{timestamp}}
Exported with
Newman v{{version}}
 
 
Total
Failed
Passed Percentage
{{#with summary}} {{#with stats}}
Iterations
{{iterations.total}}
{{iterations.failed}}
Requests
{{requests.total}}
{{requests.failed}}
Prerequest Scripts
{{prerequestScripts.total}}
{{prerequestScripts.failed}}
Test Scripts
{{testScripts.total}}
{{testScripts.failed}}
Assertions
{{assertions.total}}
{{assertions.failed}}
**
{{percent assertions.total assertions.failed}}
** {{/with}}
 
Total run duration
{{duration}}
Total data received
{{responseTotal}} (approx)
Average response time
{{responseAverage}}
 
Total Failures
{{failures}}
{{/with}}

Requests

{{#each aggregations}} {{#if parent.name}}

{{> aggregations}}
{{else}} {{> aggregations}} {{/if}} {{/each}}

{{#*inline "aggregations"}} {{#each executions}}

{{#with request}} {{#if description.content}}
Description
{{description.content}}
 
{{/if}}
Method
{{method}}
URL
Request Headers
{{headers}}
Request Body
{{body}}
{{/with}}
 
Mean time per request
{{mean.time}}

Mean size per request
{{mean.size}}

 

Total passed tests
{{cumulativeTests.passed}}
Total failed tests
{{cumulativeTests.failed}}

 

Status code
{{response.code}}


Response Headers
    {{#each response.header}}
  • {{this.key}}: {{this.value}}
  • {{/each}}
 


Response body
{{response.body}}

{{#if assertions.length}}
 
Tests
{{#each assertions}}{{/each}}
NamePass countFail count
{{this.name}}{{this.passed}}{{this.failed}}
{{/if}}
{{/each}} {{/inline}} Or If you share me your personal mail, i ll share my custom template.
kunagpal commented 6 years ago

@jayanthktn The handlebars.registerHelper call has to go within the custom reporter code, not in the script tags in the template. A sample of the file I was referring to can be found here: https://github.com/postmanlabs/newman/blob/develop/lib/reporters/html/index.js.

Using the steps mentioned here: https://www.getpostman.com/docs/postman/collection_runs/command_line_integration_with_newman#custom-reporters, you can build your own custom reporter that works with percent calculations as well.

jayanthktn commented 6 years ago

Hi Kunagpal,

I am not much aware of handlebars, i have only one .hbs file. do i need to change in hbs file or need to write separate js file.

Thanks, Jayanth

kunagpal commented 6 years ago

@jayanthktn The .hbs file will contain just the template content (of the form {{...}}). The actual logic to will have to go into a separate JS file. A starter template can be picked up from https://github.com/postmanlabs/newman/blob/develop/lib/reporters/html/index.js. Once you've added the handlebars.registerHelper snippet in this file, you can publish these files as a custom reporter, which can then be used within Newman.

jayanthktn commented 6 years ago

Thanks kunagpal. It's worked.