Newman crashes on a postman 502, does not provide details to see that it was a 502 page.

Using Newman as a lib in one of our test runners, we've observed that the Postman get-collection endpoint intermittently responds with a 502 HTML page. This crashes Newman, which always expects a JSON response. In addition, the Newman code does error masking, which made it very hard to understand that it was a 502 response coming back from Postman servers in the first place.

Began with a (redacted) stack trace like this:

Error running collection: 18503-xxxxxxx environment: 435183-xxxxxxx with error: Error: could not load environment
  Unexpected token '<' at 1:1
<html>
^
Error: could not load environment
  Unexpected token '<' at 1:1
<html>
^
    at done (/var/lib/jenkins/workspace/my-project/node_modules/newman/lib/run/options.js:119:40)
    at /var/lib/jenkins/workspace/my-project/node_modules/newman/lib/run/options.js:79:24
    at Request._callback (/var/lib/jenkins/workspace/my-project/node_modules/newman/lib/util.js:186:28)
    at Request.self.callback (/var/lib/jenkins/workspace/my-project/node_modules/postman-request/request.js:268:12)
    at Request.emit (events.js:315:20)
    at Request.EventEmitter.emit (domain.js:483:12)
    at Request.<anonymous> (/var/lib/jenkins/workspace/my-project/node_modules/postman-request/request.js:1526:10)
    at Request.emit (events.js:315:20)
    at Request.EventEmitter.emit (domain.js:483:12)
    at IncomingMessage.<anonymous> (/var/lib/jenkins/workspace/my-project/node_modules/postman-request/request.js:1397:12)
    at Object.onceWrapper (events.js:421:28)
    at IncomingMessage.emit (events.js:327:22)
    at IncomingMessage.EventEmitter.emit (domain.js:483:12)
    at endReadableNT (_stream_readable.js:1220:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)

After adding some code that adds the raw response body to the original error and embeds the original error in the masked error, was able to determine that Postman API was responding with a generic 502 HTML response:

<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
</body>
</html>

Newman Version (can be found via newman -v): 5.3
OS details (type, version, and architecture):
Are you using Newman as a library, or via the CLI? Library
Did you encounter this recently, or has this bug always been there: Seems to be happening more frequently recently
Expected behaviour: Newman provides details necessary to debug unexpected Postman API responses
Command / script used to run Newman: proprietary, but see newman stacktrace below
Sample collection, and auxiliary files (minus the sensitive details): n/a
Screenshots (if applicable):

Submitting a separate support request to determine why Postman responds intermittently with the 502, but submitting this issue to highlight Newman bug where Newman always expects JSON but the Postman API sometimes responds with non-JSON.

My above-referenced fork https://github.com/Genability/newman-fixes/pull/1/files could be one way of at least surfacing to the user what the raw response is, but recommend the Newman lib handle this more gracefully, perhaps by understanding what the root cause of the 502 is, when Postman responds with HTML instead of JSON, or more full original request/response introspection capabilities.

postmanlabs / newman

Newman crashes on a postman 502, does not provide details to see that it was a 502 page. #3153