HTTP request callback function inside reporter not called

[vdespa]

1. Newman Version (can be found via newman -v): 4.5.5
2. OS details (type, version, and architecture): macOS
3. Are you using Newman as a library, or via the CLI? CLI

I am trying to use the node request library from within a Newman reporter. Below is a sample code:

    newman.on('beforeDone', () => {
      console.log('before');

      var request = require('request');
      request('http://httpbin.org/get', function (error, response, body) {
        console.log('error:', error); // Print the error if one occurred
        console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
        console.log('body:', body); // Print the HTML for the Google homepage.
      });

      console.log('after');
    });

Maybe I am missing something, but the console.log statements inside the callback function are not called. Only before and after is printed.

Do you have any idea why?

[codenirvana]

@vdespa is execution getting terminated before request callback?

I can't think of any other reason for this issue. You can see we are doing a similar request in beforeDone callback in this reporter: https://github.com/postmanlabs/newman-reporter-remote/blob/develop/lib/index.js#L431

[vdespa]

Yes, it looks that Newman is not waiting for the callback and terminating the execution. When enabling HTTP debugging I can see that the request is being sent.

I had looked into other reporters that use HTTP and they share the same issue.

Thank you for the link, I will look at it in the next days.

[codenirvana]

@vdespa can you share the link to the reporter (or a sample reporter) with this behavior, will have a look.

[vdespa]

Thank you for looking into this.

Here is the index.js file:

function MyNewmanReporter (newman, reporterOptions, collectionRunOptions) {

    newman.on('beforeDone', () => {
      console.log('before');

      var request = require('request');
      request('http://httpbin.org/get', function (error, response, body) {
        console.log('error:', error); // Print the error if one occurred
        console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
        console.log('body:', body); // Print the HTML for the Google homepage.
      });

      console.log('after');
    });

  }
  module.exports = MyNewmanReporter

I have noticed the same behavior in this reporter as well: https://www.npmjs.com/package/newman-reporter-testrail-extra

[codenirvana]

With this I got the following logged before ending the execution: (which I feel is the expected output)

before
after
error: null
statusCode: 200
body: {
  "args": {},
  "headers": {
    "Host": "httpbin.org"
  },
  "origin": "127.0.0.1",
  "url": "https://httpbin.org/get"
}

Using Newman CLI on Node v10 & v12.

[vdespa]

I had a very strange behavior: I enabled the CLI reporter and it worked. Now it works. I don't get it :)

Thank you for your help. I will close this at this moment as I can no longer replicate the problem.

[vdespa]

@codenirvana I have managed to recreate this problem in a more stable environment:

https://gitlab.com/user-questions/newman-issues-2148

I have two parallel jobs, using the same code, one running a collection with tests that pass and the other one running a collection with tests that fail.

Passing output: https://gitlab.com/user-questions/newman-issues-2148/-/jobs/323512933 Failing output: https://gitlab.com/user-questions/newman-issues-2148/-/jobs/323512932

Only when the collection test pass, will the callback be executed.

[vdespa]

@codenirvana Can you try and replicate this issue as well?

[codenirvana]

@vdespa I can confirm this happens when there's a failure (like assertion failure) in the Newman execution because the process gets terminated with exit code 1.

For now, you can use the -x, --suppress-exit-code option to avoid this behavior.

I feel for async tasks like this, beforeDone should accept a callback function e.g:

newman.on('beforeDone', (done) => {
  request('http://httpbin.org/get', (error, response, body) => {
    done();
  });
});

[vdespa]

Is there any way to do this without (-x)?

It would be great if you could provide a "best practice" example for doing this. I see many 3rd party reporters struggling with this.

[zeburek]

@codenirvana Is there any update on this issue? I'm also faced issue described in https://github.com/postmanlabs/newman/issues/2148#issuecomment-543075798

I'm writing new reporter for quite new TMS and this looks like a very unexpected behaviour. I also have lots of async request, which are correctly served when all tests are passed and not - when something is failed.

[arirarira]

hello guys, i have this code :

let responseIGH = pm.response.json(); console.log(responseIGH); pm.collectionVariables.set("igh-file", JSON.stringify(responseIGH));

// Qase: 5005 (() => { const requestBody = JSON.parse(pm.request.body); pm.test(Ready to test IGH - Start Date: ${requestBody.startDate}, End Date: ${requestBody.endDate}, Variation: ${requestBody.variation}, function () { pm.response.to.have.status(400); }); })();

// Qase: 5100 (() => { const responseIGH = pm.response.json(); let totalIncome = 0; let allIncome = 0;

for (let i = 0; i < responseIGH.length; i++) {
    totalIncome += responseIGH[i].totalNetIncome;
    allIncome +=
        responseIGH[i].casualIncome +
        responseIGH[i].manualMixIncome +
        responseIGH[i].lostTicketIncome;
}

pm.test(`Validasi Gabungan Income (Income Casual, Income Manual Mix, Casual harian, Denda) dengan Total Income. 
    Gabungan Income ${allIncome} & Total Income  ${totalIncome}`, function () {
    pm.expect(totalIncome).to.eql(allIncome);
});

})();

// Qase: 5005 is fail case and // Qase: 5100 is passed case.

after run that code, qase just read my // Qase: 5005 case. so my case at // Qase: 5100 read as fail case. how i can different // Qase: 5005 and // Qase: 5100 ?