Support for reporters Postman v10 CLI

[w4dd325]

Is there an existing request for this feature?

• [X] I have searched the existing issues for this feature request and I know that duplicates will be closed

Is your feature request related to a problem?

Newman allows the use of custom reporters for example htmlextra . My understanding is that the new CLI in Postman v10 does not allow the use of these reporters.

Describe the solution you'd like

The ability to use custom reporters, like htmlextra with the new v10 CLI.

Describe alternatives you've considered

No response

Additional context

No response

[malvikach]

Hello James @w4dd325

Using Postman CLI, all your runs data will be available in Postman, providing you a central reporting at a single place. Read more about it here if you are interested. However, I would like to understand your feedback and thoughts on this.

Would you be up for a discussion about the reporter?

I would love a short discussion with you. Feel free to drop me a note on malvika.chaudhary@postman.com or check my calendar and block some time using this link.

[w4dd325]

@malvikach Sorry been a little swamped lately, I will try and get some time soon.

[marioleed]

My team would also, very much, like to see support for reporters so that we can use both htmlextra reporter (as we're used to) and the History centralised in Postman.

Postman CLI is nicer (CLI logging, CI/CD experience, centralised reporting and history), but the only thing that stops us from using it is the lack of reporters.

Also, for some reason we don't see body and headers (Request/Response headers unavailable, Request/Response body unavailable) in History (see image) which also stops us from using it.

[malvikach]

Hello @marioleed 👋

Thanks a lot for your detailed feedback! Both these items are in our roadmap for Postman CLI. We are not storing headers/ and response bodies yet into history, but this is something we are closely looking at.

I would be happy to stay connected with you as we solve both these items and get some feedback from you and maybe even some early trials. Let me know if you would be interested in this.

If it works, I can drop you a note in a few weeks as we get closer to solving the above-mentioned problems.

Cheers! Malvika

[marioleed]

@malvikach I'd love to assist in any way I can, thanks 🙂 Just let me know.

[dliberatore-th]

I would also like to use test reporting. My goal is to ingest results into TeamCity such that they appear like other tests and test metrics can be recorded.

TeamCity has an XML report processing feature that supports many useful report formats. https://www.jetbrains.com/help/teamcity/xml-report-processing.html

They can also read specially formatted service messages that you output to the console. https://www.jetbrains.com/help/teamcity/service-messages.html#Reporting+Tests

You can even add lots of custom metadata. https://www.jetbrains.com/help/teamcity/reporting-test-metadata.html

As a last resort, custom html report tabs can be added to build results. This probably requires that customization/scripting is reduced as far as possible to be integrated. https://www.jetbrains.com/help/teamcity/including-third-party-reports-in-the-build-results.html

It would be superb if there was linkage back to Postman client somehow, but I would be happy with simple file output that can be parsed by CICD.

[anhhanuman]

hi @malvikach About this, can you let consumers know where is the roadmap for Postman CLI? We want this the following as well

Thanks a lot for your detailed feedback! Both these items are in our roadmap for Postman CLI. We are not storing headers/ and response bodies yet into history, but this is something we are closely looking at.

Yeah we also need the support for reporter feature which we can show the client

[mayank-wpay]

Hey @malvikach any update on when this feature will be rolled out for consumers. Would be great to have custom reporters embedded right within Postman cli... Having integrations with CI/CD tools like bitbucket or Azure will make it easy to upload report artefacts to share with stakeholders.

Having a custom reporter like html makes it easier even for non-tech business person to visualise the test outcomes.

Another aspect, more of a turn off factor is run summary requires login to postman to view ex: _run data in Postman at: https://go.postman.co/workspace/*****/run/***/_** , which again can be overridden with reporters generating reports in local/cloud system like newman and share/publish it to wider audience.

[malvikach]

@mayank-wpay @anhhanuman @marioleed @dliberatore-th

We are looking at the reporter's support and the other problems mentioned in this ticket. Particularly the below aspect stands out for me:

Another aspect, more of a turn off factor is run summary requires login to postman to view ex: run data in Postman at: https://go.postman.co/workspace/*****/run/*****/ , which again can be overridden with reporters generating reports in local/cloud system like newman and share/publish it to wider audience.

Would any of you be available to spend some time discussing your workflows? I am keen to understand this so we can provide the reporter's support with Postman CLI.

Please drop me a note at malvika.chaudhary@postman.com or you can block some time with me using my calendar link here.

[marioleed]

Any progress on this? I don't know if you got any response regarding the workflows, @malvikach.

[agilenut]

We would also like to use Postman CLI but we need reporters so that we can send data to our Azure DevOps Pipelines. Azure DevOps is the central source of truth for us and it is important that we be able to store test artifacts with our other build artifacts and have the test results readable within Azure DevOps.

[pauliehyland]

Also very interested in this, especially now that the scratchpad is being deprecated, which is forcing our collections to the cloud. I'm aware that we could download the collections for use by newman (and the custom reporters), but being able to do all this using the Postman CLI would be ideal.

[webteam-eandl]

For me, I have been discussing the aspect and reporting of Postman CLI test results also. From a QA point of view (this is who primarily uses Postman where I work) we want to be able to schedule runs using the CLI to use our API credits and then report to a "JUnit-style XML file" - which is possible with Newman.

The reason is - we manage all our test activities with a single test management product. Manual testing, Test runs, and Automation UI runs (using Playwright) are all then in one place - and that is where we would want to report PLaywright CLI test runs also.

[benm912]

Agree that this would be really useful. I love how using Postman CLI you get the runs in Postman, the results are much easier to read, but you can only see these if you are part of the team and have a Postman account. Newman is great with its reporting, and using the .xml file we can create bugs directly off failed tests, but the reporting a bit messy.

[goddessbest83]

The ability to generate a report is very necessary. If you use newman, Jenkins could create a report with the -r option and send a noti to the slack. You want to use the same function when you use postman cli. I hope the function is added quickly.

[meghanbissonnette-mfj]

Any progress on this feature? We are starting to integrate our Postman collections into our CI/CD processes, but the lack of reporters for the Postman CLI is becoming an issue for us.

[GrayedFox]

Although I think "me too" comments are generally far inferior to a good old thumbs up, I've landed here after searching the Postman forums for "how can I export data from postman-cli collection runs" - even if I can't use a custom reporter, if postman-cli can produce a JSON or other output after a collection run, that would be a great start.

Does anyone know how to do that outside of just piping the results of running a collection? Having to do it that way and transforming and massaging all that output myself is what I'm trying to avoid.

[talha-akyoldas-alg]

As a team we are on the Professional plan, but unfortunately we can't use postman-cli due to reporting limitations, instead we have to use Newman in CI/CD (although there are some bugs here). In the new version, there is a nice feature like Package Library, but it is useless for us because Newman doesn't support it and we can't use postman-cli because of the reporting inadequacies. I think this issue affects users negatively in many ways. The user should be able to receive reports in different types of format (JSON, XML, etc.) individually or collectively and have the ability to create custom reports.

[mminklet]

Also have tried and failed to integrate Postman into our e2e testing suite due to seemingly bizarre and undocumented differences in variable scope, and then no way to actually debug the problem because of reporting inadequacies.