postmanlabs / newman

Newman is a command-line collection runner for Postman
https://www.postman.com
Apache License 2.0
6.88k stars 1.16k forks source link

Newman CLI sometimes generates JUnit report that is not compliant with JUnit.xsd #1582

Closed Kiran-B closed 6 years ago

Kiran-B commented 6 years ago
  1. Newman Version (can be found via newman -v): 3.9.4

  2. OS details (type, version, and architecture): macOS 10.13.5 64-bit Node v10.4.1

  3. Are you using Newman as a library, or via the CLI? CLI (installed globally via npm -g command )

  4. Did you encounter this recently, or has this bug always been there: Not sure whether it was always been there.

  5. Expected behaviour: JUnit XML report generated should be compliant with JUnit.xsd

  6. Command / script used to run Newman: newman run ./API/MyProject.postman_collection.json --reporters html,junit,cli --reporter-junit-export ./NewmanOutput/90/Junit.xml --reporter-html-export ./NewmanOutput/90/index.html -e ./API/MyProject.postman_environment.json

  7. Sample JUnit.xml generated :

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="MyProject" time="4.779000000000001">
  <testsuite name="GetCity" id="1f2b6fcf-0239-4dea-835f-136e355e269e" tests="1" failures="0" errors="0" time="0.482">
    <testcase name="Status code is 200" time="0.482"/>
  </testsuite>
</testsuites>
  1. Additional Note: The issue is random. It depends on the total time taken for the test suite to execute. It works when time attribute in the XML for node 'testsuites' is expressed using 3 decimal positions like: <testsuites name="MyProject" time="3.935"> It fails when the time contains a large string (4.779000000000001) like shown above. <testsuites name="MyProject" time="4.779000000000001">
kunagpal commented 6 years ago

@Kiran-B Thanks for reporting this, a fix has been added in #1583 😄 The fix will be rolled out with the next Newman release.

mgortzak commented 6 years ago

@kunagpal, do you know when the next Newman release will be? I'm getting the same error and I'd rather not wait a few months until this is resolved.

kunagpal commented 6 years ago

@mgortzak @Kiran-B This has been fixed in Newman v3.10.0 😄

mgortzak commented 6 years ago

@kunagpal, I've ran 5 jobs, but still no error (or any other error for that matter). So it's fixed 👍 .