postmanlabs / newman

Newman is a command-line collection runner for Postman
https://www.postman.com
Apache License 2.0
6.88k stars 1.16k forks source link

v3.6.0 no longer has `summary.run.executions.response.body` #1037

Closed deyhle closed 7 years ago

deyhle commented 7 years ago
  1. Newman Version (can be found via newman -v): 3.6.0
  2. OS details (type, version, and architecture): macOS 10.12.5 / node.js v6.10.2
  3. Are you using Newman as a library, or via the CLI? Library
  4. Did you encounter this recently, or has this bug always been there: since 3.6.0
  5. Expected behaviour: Until the latest release (up to 3.5.*) the summary.run.executions objects would include response with a property body. This was the response body as string. Since the Update to 3.6.0 the body property (summary.run.executions[0].response.body) is undefined. There seems to be no way to access the response body now?
kunagpal commented 7 years ago

@deyhle The v3.6.0 equivalent is summary.run.executions[0].response.text().

For more details, please see http://www.postmanlabs.com/postman-collection/Response.html

deyhle commented 7 years ago

Thanks for the quick response. However, summary.run.executions[0].response.text() also returns undefined, summary.run.executions[0].response.json() (the response is JSON) fails with the following error:

JSONError: Unexpected token u in JSON at position 0 at Object.parse (native) at parse (.../node_modules/liquid-json/lib/index.js:36:21) at Object.parse (.../node_modules/liquid-json/lib/index.js:58:20) at Object.json (.../node_modules/postman-collection/lib/collection/response.js:276:22)

(apparently it tries to JSON.parse undefined)

When downgrading to newman v3.5.2 it works and the body parameter is there. I just JSON.parsed the summary.run.executions[0].response.body and everything was fine. It appears that somewhere in the current version the body is swallowed, when even the accessor functions text() and json() can't read it.

kunagpal commented 7 years ago

@deyhle Could you share the collection - environment pair that causes this? Also, could you see if summary.run.executions[0].response.stream.toString() works correctly?

deyhle commented 7 years ago

Alright, here is my simplified code. summary.run.executions[0].response.stream is also undefined.

const newman = require('newman');

const collection = {
  "id": "020f340c-6dc8-0c9a-7ea9-76dde03b6a44",
  "name": "postmantest",
  "description": "",
  "order": [
    "0aae62d7-1ea7-0f85-31bf-3286e609206d"
  ],
  "folders": [],
  "timestamp": 1495454706474,
  "owner": "91456",
  "public": false,
  "requests": [
    {
      "id": "0aae62d7-1ea7-0f85-31bf-3286e609206d",
      "headers": "Content-Type: application/json\n",
      "url": "https://datamanager.cachena.entrecode.de/api/5807154b/postmantest",
      "pathVariables": {},
      "preRequestScript": null,
      "method": "POST",
      "collectionId": "020f340c-6dc8-0c9a-7ea9-76dde03b6a44",
      "data": [],
      "dataMode": "raw",
      "name": "https://datamanager.cachena.entrecode.de/api/5807154b",
      "description": "",
      "descriptionFormat": "html",
      "time": 1495454782004,
      "version": 2,
      "responses": [],
      "tests": null,
      "currentHelper": "normal",
      "helperAttributes": {},
      "rawModeData": "{\"text\":\"test\"}"
    }
  ]
};

new Promise((resolve, reject) => newman.run({
  collection,
  environment: {},
  globals: {},
}, (error, summary) => error ? reject(error) : resolve(summary)))
.then((summary) => {
  return JSON.parse(summary.run.executions[0].response.body); // this used to work
  summary.run.executions[0].response.text(); // undefined
  summary.run.executions[0].response.json(); // JSONError: Unexpected token u in JSON at position 0
  summary.run.executions[0].response.stream.toString(); // Cannot read property 'toString' of undefined
})
.catch((e) => {
  console.error(e.message);
});

The called API responds with HTTP 201 and a JSON response body.

kunagpal commented 7 years ago

@deyhle Thanks, I was able to reproduce the issue with the above.

deyhle commented 7 years ago

Any updates on this? When can we expect a fix?

czardoz commented 7 years ago

@deyhle we're a little swamped on some work on postman runtime (https://github.com/postmanlabs/postman-runtime/pull/266 in case you're curious), but we'll try to push out a fix early next week.

In the meantime, feel free to downgrade to v3.5.2 if this bug is a show-stopper for you :(

kunagpal commented 7 years ago

@deyhle A fix for this bug has been added in #1080, and will be rolled out with the next Newman release. We'll update this thread accordingly 😄

Thanks for reporting this!

kunagpal commented 7 years ago

@deyhle This bug has been fixed in Newman v3.7.0. Do go ahead and check it out!

aksbenz commented 7 years ago

We are using the {{response.body}} in a custom html template. Which doesn't work anymore. And you cant put function calls in handlebar templates, so cant do response.text() or response.json(). Any suggestion on how we can put the json body in custom template ?

deyhle commented 7 years ago

In v3.7.0 at least response.text() and response.json() work. However, as @aksbenz noted, response.body is still undefined which can lead to problems. Would be great to just add a response.body = response.text() somewhere to fix backwards compatibility. Right now, it is definitely a breaking change, which violates Semantic Versioning.

aksbenz commented 7 years ago

As an improvement (also a workaround) can we have the handlebars-helpers library included as part of newman html reporter. This will give access to a lot more helpers in creating custom templates.

kunagpal commented 7 years ago

@deyhle @aksbenz Fixed in #1098

aksbenz commented 7 years ago

Raised pull request https://github.com/postmanlabs/newman/pull/1099 with an update to fix.

kunagpal commented 7 years ago

@aksbenz Fixed in Newman v3.7.4