reportportal / agent-js-postman

Agent to integrate Postman (based on Newman collection runner) with ReportPortal.
https://www.npmjs.com/package/@reportportal/newman-reporter-agent-js-postman
Apache License 2.0
17 stars 8 forks source link

Template strings eg. backticks not supported #60

Open jjj-holger opened 3 years ago

jjj-holger commented 3 years ago

If Postman test uses template strings like shown below, then these are not parsed correctly because regex used for detecting tests does not take backticks into account.

pm.test(`Should include ${sectorName} in the records list`, function () {
      const countries = pm.response.json().success.data.countries;
      const countryUS = countries.find
        (data => data.name === "United States");
      const responseSector = countryUS.sectors.find
        (data => data.name === sectorName);
      pm.expect(responseSector.name).to.eql(sectorName)
});
dantench commented 3 years ago

@jjj-holger this looks the same as my issue :) https://github.com/reportportal/agent-js-postman/issues/63

If you take the variable out the name, it might be alright.

jjj-holger commented 3 years ago

@dantench yeah that's the same underlying reason. I looked into it and found that the launch remains in progress forever due to code line in reporter.js file in the finishStep function: const currentStepIndex = testObj.steps.findIndex(step => step.name === testAssertion.assertion) The problem here is that the step name is taken literally from Postman tests using regex, but the assertion which it's compared to is processed by Newman. Which in my case means that the comparison is made between Should include ${sectorName} in the records list and Should include Industrial-sector in the records list. Meaning that when Postman JS agent tries to send a request to ReportPortal that "hey, that test result you sent previously is PASSED/FAILED" then it can't find any match from the previous test steps it sent before (it needs the stepID from the first request).

TLDR the best solution is to remove the variables for time being; although altering some code lines I was able to fix this permanent in progress issue (this "hack" still has some problems though).

chivekrodis commented 3 years ago

Hello @dantench @jjj-holger We will investigate this case and make a decision on it. similar issue https://github.com/reportportal/agent-js-postman/issues/63