billylam / newman-reporter-testrail

TestRail reporter for Newman
https://www.npmjs.com/package/newman-reporter-testrail
MIT License
26 stars 21 forks source link

Getting 400 for add_results_for_cases #1

Closed devlina21 closed 6 years ago

devlina21 commented 6 years ago

I am not able to get results from newman into testrail add_results_for_cases API This is my command line: newman run test.postman_collection.json -e QA_3.postman_environment.json --delay-request 50 -r testrail

This is part of the logs: Error: Server responded to https://testrail.com/testrail/index.php?/api/v2/add_results_for_cases/48461 with status code 400: {"error":"Field :results cannot be empty (one result is required)"} at Response.getBody (C:\Users\d\AppData\Roaming\npm\node_modules\newman-reporter-testrail\node_modules\http-response-object\lib\index.js:41:23) at EventEmitter.TestRailReporter.emitter.on (C:\Users\d\AppData\Roaming\npm\node_modules\newman-reporter-testrail\lib\TestrailReporter.js:58:62) at EventEmitter.emit (C:\Users\d\AppData\Roaming\npm\node_modules\newman\node_modules\eventemitter3\index.js:203:33) at done (C:\Users\d\AppData\Roaming\npm\node_modules\newman\lib\run\index.js:248:29) at C:\Users\d\AppData\Roaming\npm\node_modules\newman\node_modules\postman-runtime\lib\backpack\index.js:56:34 at PostmanCollectionRun._process (C:\Users\d\AppData\Roaming\npm\node_modules\newman\node_modules\postman-runtime\lib\runner\run.js:161:13) at PostmanCollectionRun.<anonymous> (C:\Users\d\AppData\Roaming\npm\node_modules\newman\node_modules\postman-runtime\lib\runner\run.js:166:76) at ontimeout (timers.js:386:14) at tryOnTimeout (timers.js:250:5) at Timer.listOnTimeout (timers.js:214:5)

I am also seeing this in the logs: newman warning: "testrail" reporter could not be loaded.

billylam commented 6 years ago

I would expect this if the reporter wasn't able to correctly identify test cases based on the assertions. Can you send an example of a test assertion that you have? Also make sure the cases are prepended with C. e.g.

pm.test("C226750 Status code is 200", function () {
    pm.response.to.have.status(200);
});

will work

pm.test("226750 Status code is 200", function () {
    pm.response.to.have.status(200);
});

will not.

I'm not quite sure why the "not loaded" newman warning would happen but will research to see if I can find anything.

devlina21 commented 6 years ago

I did a little more digging into the code and this resolved my issue:

class TestRailReporter {
  constructor(emitter, reporterOptions, options) {
    const results = [];
     emitter.on('beforeDone', (err, o) => {
     ...
      });
  }
}
billylam commented 6 years ago

Just adding params to the "beforeDone" event? If so I'm surprised because they're unused but feel free to file a PR.

billylam commented 6 years ago

I've added args to the method signature. Please reopen if you still are having issues.