postmanlabs / postman-app-support

Postman is an API platform for building and using APIs. Postman simplifies each step of the API lifecycle and streamlines collaboration so you can create better APIs—faster.
https://www.postman.com
5.82k stars 839 forks source link

[Feature Request] - Expose the new pm.test() tests to the 'tests' object #3943

Open Faz540 opened 6 years ago

Faz540 commented 6 years ago

Hi, Given I have the below test script in my endpoint and I'm expecting both tests to fail.

tests["HTTP Status Code = 200"] = responseCode.code === 200;
const response = JSON.parse(responseBody);
pm.test("test name", function() {
    // This is failing
    pm.expect(actual).to.equal(expected);
 });

console.log(tests);

The console will only output the first test, the "old" way of writing tests.

image

If possible, can we expose the newer "pm.test()" tests to the same tests object. Or given them their own object.

I use this to get the number of failed tests and I then send myself a Slack Message if the fail counter is greater than 0.

I got into detail about this here: https://community.getpostman.com/t/notify-slack-when-test-failures-are-found/175/8

sdnts commented 6 years ago

@Faz540 Thanks for this request. We're aiming to deprecate the old tests object, so this seems counter-intuitive to me. However, it is understandable that there might be old requests that use a combination of these kinds of assertions. @shamasis @kamalaknn Thoughts?

Faz540 commented 6 years ago

@madebysid Please tell me there will be a way to monitor the true/false values of the new "pm.test()" tests in the future? Ofcourse, other than viewing the Tests tab in the UI.

ghost commented 6 years ago

I agree, pm.test() returning boolean status or some object to retrieve test status would be super-userful

TesterSpike commented 6 years ago

I use the pass/fail states of the tests to determine whether to continue travelling down one path or another. It's doesn't feel DRY writing the tests to get the messaging and then rewriting them in a slightly different way to get the pass/fail state so I can have a functioning test suite. Without this feature postman becomes a lot less useful to me.

gsperk commented 5 years ago

I need this too. It's a pain to re-evaluate for pass/fail state to do some control logic for the overall collection path.

Sophos06 commented 5 years ago

Really looking forward to - pm.test.result() or any boolean returned by pm.test()... Until then, I did not even believed it was not possible. Painful.