spenceclark / newman-reporter-json-summary

A Newman JSON Reporter that strips the results down to a minimum
8 stars 7 forks source link

Reporter fails to handle failures within pre-request scripts #12

Open jhulme opened 2 years ago

jhulme commented 2 years ago

Reporter fails to handle failures that occur within pre-request scripts.

The following error message is returned.

/home/john/forge/postman-collections/node_modules/newman-reporter-json-summary/index.js:27
                'Name': failure.parent.name,
                                       ^

TypeError: Cannot read property 'name' of undefined
    at /home/john/forge/postman-collections/node_modules/newman-reporter-json-summary/index.js:27:40
    at Array.forEach (<anonymous>)
    at createSummary (/home/john/forge/postman-collections/node_modules/newman-reporter-json-summary/index.js:23:26)
    at EventEmitter.<anonymous> (/home/john/forge/postman-collections/node_modules/newman-reporter-json-summary/index.js:70:37)
    at EventEmitter.emit (/home/john/forge/postman-collections/node_modules/eventemitter3/index.js:203:33)
    at done (/home/john/forge/postman-collections/node_modules/newman/lib/run/index.js:314:29)
    at /home/john/forge/postman-collections/node_modules/postman-runtime/lib/backpack/index.js:58:34
    at PostmanCollectionRun._process (/home/john/forge/postman-collections/node_modules/postman-runtime/lib/runner/run.js:163:13)
    at PostmanCollectionRun.<anonymous> (/home/john/forge/postman-collections/node_modules/postman-runtime/lib/runner/run.js:169:76)
    at listOnTimeout (node:internal/timers:556:17)
    at processTimers (node:internal/timers:499:7)

Taking a closer look at the failure object returned, you can see that when a failure occurs within a pre-request script the parent is left undefined.

FAILURE: {
  error: SerialisedError {
    errno: -3008,
    code: 'ENOTFOUND',
    syscall: 'getaddrinfo',
    hostname: 'auth-incorrect.test.com',
    name: 'Error',
    message: 'getaddrinfo ENOTFOUND auth-incorrect.test.com',
    stack: 'Error: getaddrinfo ENOTFOUND auth-incorrect.test.com\n' +
      '    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:69:26)',
    checksum: '31d5dc918783eb7f70b5f40b233e2629',
    id: 'removed',
    timestamp: 1633441749292,
    stacktrace: [ [Object] ]
  },
  at: 'request',
  source: PostmanItem {
    id: 'removed',
    request: PostmanRequest {
      url: [PostmanUrl],
      headers: [HeaderList],
      method: 'POST',
      body: [PostmanRequestBody]
    },
    responses: PostmanPropertyList {
      members: [],
      reference: {},
      Type: [Function]
    },
    events: PostmanEventList { members: [], reference: {}, Type: [Function] }
  },
  parent: undefined,
  cursor: {
    ref: '30d3f642-02b5-41db-a034-42fc3253e42a',
    length: 43,
    cycles: 1,
    position: 1,
    iteration: 0,
    scriptId: 'removed',
    execution: 'removed',
    httpRequestId: 'removed'
  }
}

Assuming that simply discounting these failures when building the summary object would be sufficient. Unless there is a particular need to include these failures within the final report summary?

ox-rjaumain commented 1 year ago

Using pre-request scripts in collections as well, raising the same error.

/home/user/project/node_modules/newman-reporter-json-summary/index.js:25
                'Name': failure.parent.name,
                                       ^
TypeError: Cannot read property 'name' of undefined
    at /builds/openxtrem/mediboard/node_modules/newman-reporter-json-summary/index.js:25:40
    at Array.forEach (<anonymous>)
    at createSummary (/builds/openxtrem/mediboard/node_modules/newman-reporter-json-summary/index.js:22:26)
    at EventEmitter.<anonymous> (/builds/openxtrem/mediboard/node_modules/newman-reporter-json-summary/index.js:68:37)
    at EventEmitter.emit (/builds/openxtrem/mediboard/node_modules/eventemitter3/index.js:203:33)
    at done (/builds/openxtrem/mediboard/node_modules/newman/lib/run/index.js:314:29)
    at /builds/openxtrem/mediboard/node_modules/postman-runtime/lib/backpack/index.js:58:34
    at PostmanCollectionRun._process (/builds/openxtrem/mediboard/node_modules/postman-runtime/lib/runner/run.js:160:13)
    at PostmanCollectionRun.<anonymous> (/builds/openxtrem/mediboard/node_modules/postman-runtime/lib/runner/run.js:166:76)
    at listOnTimeout (internal/timers.js:557:17)
    at processTimers (internal/timers.js:500:7)

I agree with @jhulme suggestion to discard those failures in the first place.