postmanlabs / newman

Newman is a command-line collection runner for Postman
https://www.postman.com
Apache License 2.0
6.82k stars 1.15k forks source link

Updated dependencies #3183

Closed KiraElric007 closed 8 months ago

KiraElric007 commented 8 months ago

Sourced from postman-runtime's changelog.

7.36.0: date: 2023-11-18 new features: - >- GH-1336 Added support for fetching execution location context through script chores: - Updated dependencies

7.35.0: date: 2023-11-02 new features: - GH-1354 Added support for skipping request execution through script

7.34.0: date: 2023-10-19 new features: - GH-1340 Added support for ASAP authentication chores: - Updated dependencies

Sourced from postman-collection's changelog.

4.3.0: date: 2023-11-18 new features: - GH-1339 Added getPath method on Item chores: - Updated dependencies

codecov-commenter commented 8 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (a612fbe) 89.96% compared to head (396eed3) 89.96%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #3183 +/- ## ======================================== Coverage 89.96% 89.96% ======================================== Files 21 21 Lines 1146 1146 Branches 347 347 ======================================== Hits 1031 1031 Misses 115 115 ``` | [Flag](https://app.codecov.io/gh/postmanlabs/newman/pull/3183/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Postman) | Coverage Δ | | |---|---|---| | [cli](https://app.codecov.io/gh/postmanlabs/newman/pull/3183/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Postman) | `82.19% <ø> (ø)` | | | [integration](https://app.codecov.io/gh/postmanlabs/newman/pull/3183/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Postman) | `40.92% <ø> (ø)` | | | [library](https://app.codecov.io/gh/postmanlabs/newman/pull/3183/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Postman) | `58.55% <ø> (ø)` | | | [unit](https://app.codecov.io/gh/postmanlabs/newman/pull/3183/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Postman) | `74.25% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Postman#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

filipeandre commented 8 months ago

Hey, guys, the pr looks good. May you five me permissions to merge prs... Looks like this feature of postman is getting old > 3 months and the non merge of this pr makes this project lag beind when it shoudn't, at least when the reason are the postman vendor libs.

filipeandre commented 8 months ago

but the test skip is not related with the PR itself, but rather the node version. have we a pr to fix the incompatibility with this test and node v20.10 ?

KiraElric007 commented 8 months ago

@vedkribhu The skipped test case, fails for timeout >= 7 ms for node versions < 20.10.0. This node PR adds a performance improvement in node v20.10.0 resulting in the test failing for timeout >= 4 ms. There seems to be a race condition that needs to be fixed in postman-runtime/postman-sandbox library.

The test case has been testing an edge case which passes, leaving out all the other cases when it actually fails. Since this is happening in develop as well, we can look into this in a separate PR. Let us not block this PR.

KiraElric007 commented 8 months ago

I am attaching below screenshots of test case execution in different node versions with different timeout (All tests are run on develop branch [head commit]).

Test passing in node v16.10.0 with 5ms timeout:

Screenshot 2023-12-20 at 2 30 01 PM

Test failing in node v16.10.0 with 10ms timeout:

Screenshot 2023-12-20 at 2 28 17 PM

Test passing in node v20.10.0 with 3ms timeout:

Screenshot 2023-12-20 at 2 32 02 PM

Test failing in node v20.10.0 with 5ms timeout:

Screenshot 2023-12-20 at 2 31 11 PM

The above screenshot proves that the issue is existing in develop and not introduced because of this PR. We can deep dive into the root cause of this issue and fix it in a separate PR.

cc: @aman-postman