Remove collection run limit for non scheduled cloud based runs.

[byrdman]

Is there an existing request for this feature?

• [X] I have searched the existing issues for this feature request and I know that duplicates will be closed

Is your feature request related to a problem?

My development cycle has me running collections from within Postman multiple times during the day, I can quickly reach the new 25 max run limit in a morning. I don't understand why I'm limited to the # of collection runs I can do locally but not if I use the cloud runner. My API isn't accessable to the cloud runner. This has ruined a great tool. The only way I can continue to use Postman is to either upgrade to the Pro license or do all my runs using the CLI (which removes many of the benefits of using Postman).

Describe the solution you'd like

Remove the limit for non-scheduled cloud runs. It doesn't cost you any resources for us to run collections 1000 times on our local hardware (which you now limit to 25 times per month) vs doing scheduled runs from your cloud (which doesn't work in our environment).

Describe alternatives you've considered

MIgrating off of Postman Writing my own tool based on the CLI.

Additional context

All of the "push everything to the cloud" has ruined the great product that Postman once was. It took over a year to get the performance problem of importing large collections fixed since it was uploading it to your cloud. There was no reason for it to be uploaded to the cloud since Postman doesn't play nice with our git system (it's not publicly accessible therefore Postman can't talk to it so there's no reason to collaborate in the cloud).

[byrdman]

I just found the blog post announcing the change. More than a month notice for folks to change their processes would have been a nice thing to do, some of us have built processes around your tool and not everyone's environment is available on the internet. My organization can't justify $1200/user/year.

[malvikach]

@byrdman Thanks for writing to us and sharing your concerns.

Please send us more details at help@postman.com so we can understand your situation better and support you. Our goal is to enable your workflows as seamlessly as possible while you get time to respond to the change. If you wish to discuss this instead, please use this calendly link to schedule some time with me. We would like to understand your current workflows and how you use Runner.

The changes will be effective on March 15 if you are on our basic/professional plan and on Feb 15th if you are on the free plan (the limit will be effective post your plan renewal happens i.e. existing plans or plans purchased before March 15th will have unlimited runs until their renewal date) .

Meanwhile, I wanted to inform you that Postman CLI integrates well with Postman, such that: the commands can be quickly generated and used and once your runs are completed, they are available within Postman so you can examine the results. You can read more about Postman CLI on our blog here. Let us know if this is helpful.

[anandvarkeyphilips]

I was pushing heavily my team to see benefits of Postman. Now this I see postman differently!

Thread on community forum! https://community.postman.com/t/250-collection-runs-per-month-on-professional/43946/41

[byrdman]

I'll take a look at if using the CLI will work for us. Seems stupid to have to run it outside of the app and then can see the results within the app. We use zero of the cloud features because they don't add any value to our processes. For over a year the new cloud features slowed us down because our collections are so large your cloud kept timing out while trying to load the collections. Why should I have to generate CLI commands, run those, then look at the results in the app. It's so much lower friction to just click Run collection. I may take you up on the having a chat, let me play with the CLI a bit first.

On Mon, Feb 20, 2023 at 3:09 PM Malvika Chaudhary @.***> wrote:

@byrdman https://github.com/byrdman Thanks for writing to us and sharing your concerns.

Please send us more details at @.*** so we can understand your situation better and support you. Our goal is to enable your workflows as seamlessly as possible while you get time to respond to the change. If you wish to discuss this instead, please use this calendly link https://calendly.com/malvika-chaudhary/v10-feedback to schedule some time with me. We would like to understand your current workflows and how you use Runner.

The changes will be effective on March 15 if you are on our basic/professional plan and on Feb 15th if you are on the free plan (the limit will be effective post your plan renewal happens i.e. existing plans or plans purchased before March 15th will have unlimited runs until their renewal date) .

Meanwhile, I wanted to inform you that Postman CLI integrates well with Postman, such that: the commands can be quickly generated and used and once your runs are completed, they are available within Postman so you can examine the results. You can read more about Postman CLI on our blog here https://blog.postman.com/introducing-the-postman-cli-to-automate-your-api-testing/. Let us know if this is helpful.

— Reply to this email directly, view it on GitHub https://github.com/postmanlabs/postman-app-support/issues/11744#issuecomment-1437546096, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASBINTHP5JSD2IW5YZDM5DWYPMQ3ANCNFSM6AAAAAAVB5XRNA . You are receiving this because you were mentioned.Message ID: @.***>

[byrdman]

So calling using the cli works, though as I said it's cludgy, breaks the overall workflow, and the really big thing is that I really don't see what the difference is between "Run manually" vs "Run in CLI" is (other than for 1 I can just click a button and the other I have to copy a command out to a command window). They both upload the run to the cloud. The biggest difference is that "Run manually" logs data into the console log that I can use for debugging things not working. I have collections that have thousands of requests broken into hundreds of folders. Very often during the development process I'll run a folder that holds a dozen or so requests walking through some scenario. Doing this allows me to then examine what's in the console. My more common scenario is to run a folder that setups data in a certain way, then I'll individually run a much smaller set so I can more easily investigate what is sent/returned on those requests.

On Thu, Feb 23, 2023 at 5:11 PM Sam Byrd @.***> wrote:

I'll take a look at if using the CLI will work for us. Seems stupid to have to run it outside of the app and then can see the results within the app. We use zero of the cloud features because they don't add any value to our processes. For over a year the new cloud features slowed us down because our collections are so large your cloud kept timing out while trying to load the collections. Why should I have to generate CLI commands, run those, then look at the results in the app. It's so much lower friction to just click Run collection. I may take you up on the having a chat, let me play with the CLI a bit first.

On Mon, Feb 20, 2023 at 3:09 PM Malvika Chaudhary < @.***> wrote:

@byrdman https://github.com/byrdman Thanks for writing to us and sharing your concerns.

Please send us more details at @.*** so we can understand your situation better and support you. Our goal is to enable your workflows as seamlessly as possible while you get time to respond to the change. If you wish to discuss this instead, please use this calendly link https://calendly.com/malvika-chaudhary/v10-feedback to schedule some time with me. We would like to understand your current workflows and how you use Runner.

The changes will be effective on March 15 if you are on our basic/professional plan and on Feb 15th if you are on the free plan (the limit will be effective post your plan renewal happens i.e. existing plans or plans purchased before March 15th will have unlimited runs until their renewal date) .

Meanwhile, I wanted to inform you that Postman CLI integrates well with Postman, such that: the commands can be quickly generated and used and once your runs are completed, they are available within Postman so you can examine the results. You can read more about Postman CLI on our blog here https://blog.postman.com/introducing-the-postman-cli-to-automate-your-api-testing/. Let us know if this is helpful.

— Reply to this email directly, view it on GitHub https://github.com/postmanlabs/postman-app-support/issues/11744#issuecomment-1437546096, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASBINTHP5JSD2IW5YZDM5DWYPMQ3ANCNFSM6AAAAAAVB5XRNA . You are receiving this because you were mentioned.Message ID: @.***>

[kenficara]

I was working on a demo to my organization encouraging them to use Postman, but if I say "It'll cost us $1200/seat to actually use the tool," they'll laugh me out of the room. What a sad end to a good tool.

[byrdman]

Ken, Take a look at the CLI that Malvika suggested above. The installation could be a little more enterprise friendly. I don't know many enterprises that allow random users to run a powershell script that downloads random content from the internet. I had to build a different script for my team to use to do the install. t's certainly not as frictionless as being able to just "Run Manually" and I'm waiting to hear back on what "Run Manually" provides that "Automate Runs in CLI" doesn't that justifies the limit but it does work pretty nicely. We've hundreds of thousands of hours into building a test suite around Postman and it's testing scripting. But yea, having to goto management with a $36k+ ask for a tool that we've been using for free for years with zero added benefit to us isn't going over well.

On Fri, Feb 24, 2023 at 10:12 AM Ken Ficara @.***> wrote:

I was working on a demo to my organization encouraging them to use Postman, but if I say "It'll cost us $1200/seat to actually use the tool," they'll laugh me out of the room. What a sad end to a good tool.

— Reply to this email directly, view it on GitHub https://github.com/postmanlabs/postman-app-support/issues/11744#issuecomment-1443925268, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASBINQG4LJK6IHRASKBFU3WZDMWXANCNFSM6AAAAAAVB5XRNA . You are receiving this because you were mentioned.Message ID: @.***>

[Mike-Soft]

My project paid a lots of money last year. Almost 2000$ for 12 people on basic plan. And now we will have to pay almost 5000$ so we can click on manual run? I'll be forced to find a replacement tool now. I could understand for example 1000 runs on basic and 10000 on proffesional, but generally idea for limiting this is so weird. What's next? Limit of requests sent in a month? Limit of env variables to 20 in proffesional plan? You became unpredictable tool. Great job Postman, even if we upgrade now and use your offer to have unlimited runs for some time on proffesional after a year we will be forced out. 250 runs will not be enough. No company will pay 100$ per person per month for a possibility to click a button. You are a company and you seem quite bad at doing business. Instead of having lots of users paying less you want to have few users paying a lot? You'll be amazing example of ruining business image for years to come.

[kkevindev]

25 runs for 15$ per month per user. 250 runs for 36$ per month per user.

Topkek. Nice cash grab Postman.

[xparpx]

I'm using Postman since about 5/6 years ago. I recommended this tool to my company's QA team about 2 years ago. What my company will think when I inform them that now we need to pay 30$ 5 users 12 Months? Of course, they will say me to switch to another tool. We need to run tests locally when we are debugging some requests or an entire collection. More than 100x per month. Using Postman CLI every time is not a solution for QA team.

Btw, Which tools do you suggest to replace the Postman? SOAP Ui? Rest Assured?

[fahad36hossain]

Why am I seeing run limits when just running a few tests in private workspaces?? The postman website says there are no limits for personal workspaces. Please fix this. I am not running anything using a scheduler. It's just manual runs in a personal workspace not associated with any team.

[malvikach]

@fahad36hossain The run limits apply to the personal workspaces too the same way. Each collection run is counted as a single usage irrespective of the workspace the collection is being run from. The limits reset on a monthly basis.

Could you help point to the place where you read about this?

Our learning center documentation is a good place to clarify any doubts you might have.

Please reach out to us if you have further questions at help@postman.com or set up time to talk to me.

[ironboy]

This is just silly - I used Postman for years teaching my uni students API/endpoint-testing. We can get around this by using Newman... but I feel it's just a matter of time until that gets locked down...

I will be moving to https://hoppscotch.io/ (the have an alpha of an equivalent to Newman ready) from now on. And tell my students that Postman "was the big tool" but Hoppscotch is an excellent open source clone.

A sad end indeed to what used to be "the" endpoint-testing tool, but has grown lazy, bloated and greedy.

[adityakap]

About 3 years ago, I convinced my org to subscribe to the team collaboration plan on Postman, facilitating easy sharing of workspaces and collections across teams, among other benefits. Since then postman has been billing us for 10-12 users annually. And as recently as a month ago a different group within my org joined the team and it increased to 20 members.

We primarily use Postman GUI to build basic API tests and quick data driven tests. We execute them within a Docker container using Newman on our own cloud resources. As a result, the limitations on local collection runs did not significantly affect our existing CI/CD automated testing workflows. It impacts the speed with which we can develop and test them. However, I understand that this may be temporary, considering the direction you intend to take your product.

A single update to the Newman CLI image that we pull from Docker Hub could potentially disrupt our workflows. Therefore, I have decided to halt further development of API tests on Postman. Instead, I will focus on migrating all our existing API tests to our pytest framework and explore other solutions for day-to-day API testing and collaboration.

[Winfle]

This is totally cringe. Local runs for $50 for a http tool is insane!