postmanlabs / newman

Newman is a command-line collection runner for Postman
https://www.postman.com
Apache License 2.0
6.83k stars 1.16k forks source link

Requests made by Newman are erroring (Latest version 6.2.0) #3270

Closed daveyq1 closed 2 weeks ago

daveyq1 commented 3 weeks ago
  1. Newman Version (can be found via newman -v): 6.2.0
  2. OS details (type, version, and architecture): Edition Windows 10 Business Version 22H2 OS build 19045.4651 Experience Windows Feature Experience Pack 1000.19060.1000.0 Processor 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz 1.80 GHz Installed RAM 16.0 GB (15.7 GB usable) System type 64-bit operating system, x64-based processor
  3. Are you using Newman as a library, or via the CLI? CLI
  4. Did you encounter this recently, or has this bug always been there: since v6.2.0
  5. Expected behaviour: get a 200 response
  6. Command / script used to run Newman: from batch file
    @echo off
    set postmanPath=C:\somedir
    set "postmanCollection="%postmanPath%\_test.postman_collection.json""
    call newman run %postmanCollection% ^
    --insecure
    pause
  7. Sample collection, and auxiliary files (minus the sensitive details): _test.postman_collection.json
  8. Screenshots (if applicable): image

Steps to reproduce the problem:

  1. Save the code above to a batch file
  2. run the batch file
codenirvana commented 3 weeks ago

@daveyq1 Thanks for reporting this issue. Are you facing this issue consistently?

I tried executing the shared collection, working fine for me.

image
daveyq1 commented 3 weeks ago

Hi @codenirvana yes consistently. Also confirmed with a colleague, they are seeing the same issue

g1st commented 3 weeks ago

Might be a similar issue, newman 6.2.0 now is hanging with this output until force quit:

error: Cannot read properties of undefined (reading 'exec')

Downgrading to 6.1.3 and all works fine.

newman-error
codenirvana commented 3 weeks ago

@g1st can you share a scaled-down collection that we can use to reproduce this issue?

amydoxym commented 3 weeks ago

Spent half a day troubleshooting this issue. Please note that we get the same issue and it is a collection that used to work. Just changed the version from 6.2.0 to 6.1.3 and it is back to working now.

parthverma1 commented 3 weeks ago

@amydoxym Can you share a scaled-down collection that we can use to reproduce the issue? Can you also share the node version that you are encountering this error on?

amydoxym commented 3 weeks ago

@parthverma1 Node version: 18.20.4 Request

curl --location 'https://localhost:8443/app/api/v1/clients?limit=15&offset=0' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer TOKEN' \
--header 'Cookie: JSESSIONID=3C69E588088F3293E58B6577BDBDAFA5; OAuth_Token_Request_State=19fcd077-4d66-4bc6-86e6-33735d34bdde'

Here was the test that was failing

pm.test("The response has all properties", () => {  
     const responseJson = pm.response.json(); 
     console.log(responseJson)
});
pm.test("Status code is 200", function () {
    pm.response.to.have.status(200);
});

The response was always undefined. Worth mentioning that this was failing only on requests with https. Requests using http were passing.

parthverma1 commented 3 weeks ago

@amydoxym I tried replicating the issue by running a local https server using nodejs with the following code

const https = require("https");
const fs = require("fs");
const path = require("path");

const server = https.createServer(
  {
    key: fs.readFileSync(
      path.resolve(
        "localhost.key"
      )
    ),
    cert: fs.readFileSync(
      path.resolve(
        "localhost.crt"
      )
    ),
    ca: fs.readFileSync(
      path.resolve(
        "ca.crt"
      )
    ),
  },
  (req, res) => {
    res.end(JSON.stringify({ message: "Hello World" }));
  }
);
server.on("error", (e) => {
  if (e.code === "EADDRINUSE") {
    console.error("Address in use, retrying...");
  }
});
const port = 3000;

server.listen(port, (err) => {
  console.log(`Server running at http://localhost:${port}/`);
});

and the following collection file newman.postman_collection.json and the command npx newman run newman.postman_collection.json -k but was unable to replicate the issue.

image

Since the execution failed for a localhost application, I believe it has something to do with how the local server is running. Can you share details about the web framework you are using and anything special in the way the server is setup that can help us replicate the issue.

Additionally, we can connect over email to discuss more.

MauriceS commented 3 weeks ago

I have a similar issue with 6.2.0 that only concerns https requests towards a public server (not on localhost) with a DigiCert wildcard SSL certificate. Specifying -k or --insecure did not fix the issue. It hangs at the first POST https request it does and I have to force quit it.

Changing the environment file such that the base url is http instead of https makes it work fine.

Downgrading to 6.1.3 also fixes the issue with https. That is my current workaround.

PS: Using curl https://xxx.xxxxxxxxxxxxxxx.xx/api/company.json works fine.

root@37eb535dc670:/home/node# newman run ext.json -e env.json -r cli,junit -k
newman

testSuiteExtensive

❏ Setup users
↳ check that a company can be added
  POST https://xxx.xxxxxxxxxxxxxxx.xx/api/company.json ⠋^C
parthverma1 commented 3 weeks ago

@MauriceS Can you share more information about the server to which the https request failed? You mentioned that it is a public server. Would it be possible for you to share the domain that you are trying to send the request to so that we can replicate it locally? If you are not comfortable sharing the domain publicly, we can communicate over email (api-client@postman.com) to diagnose the issue further.

jjayabal23 commented 3 weeks ago

I don't see any errors, but the newman waits indefinitely without throwing any errors.

My collection works only with version 6.1.3.

MauriceS commented 3 weeks ago

@parthverma1 I have send a simplified test that reproduces the issue to your email address.

keithboone commented 3 weeks ago

I too am having this problem with newman 6.2.0. newman hangs on first request and never gets to the next one. Ours runs on ubuntu within github CICD runner. Downgrading to newman 6.1.3 resolved our issue as well. This may have been resolved with 6.2.1, I didn't have time to verify.

keithboone commented 3 weeks ago

The server under test has a misconfigured key store that includes not just the cert chain, but also the root CA cert. This could lead to an endless loop. That may be a special case that triggers this.

AzkaNoreen6 commented 3 weeks ago

I don't see any errors, but the newman waits indefinitely without throwing any errors.

My collection works only with version 6.1.3.

yes, also for my collection It waits and then gives this

image

ECONNRESET error, while making a tls connection. Reverting to 6.1.3 fixes this.

confiq commented 3 weeks ago

We had this problem on older versions of Node. We fixed it by using Node@21. Hope it helps someone.

MauriceS commented 2 weeks ago

@parthverma1 I can confirm that the issue I was having is fixed with Newman 6.2.1, both in Node 18 and 21. Thanks!

parthverma1 commented 2 weeks ago

Glad to hear that @MauriceS. I'll be closing this issue since upgrading to 6.2.1 should solve the issues reported in this thread. Feel free to create another issue if there any issues with the new version.

AzkaNoreen6 commented 2 weeks ago

Glad to hear that @MauriceS. I'll be closing this issue since upgrading to 6.2.1 should solve the issues reported in this thread. Feel free to create another issue if there any issues with the new version. yes, with new version it is fixed.