Add an option to "Update Collection" generated from the APIs tab

[tmdoit-zz]

Is your feature request related to a problem? Please describe. When updating collections from API schema, all examples are lost - you need recreate all examples from zero - this kind of development is useless if you prefer develop collections from API spec.

Describe the solution you'd like Add option "Update Collection" next to "Generate Collection" in APIs tab

Describe alternatives you've considered After every update add manually all examples.

IMHO it's priority for APIs new feature you released.

[tmdoit-zz]

Sorry for confusion, now I see that response examples are generated from API spec.

[saivan]

But I still think this is probably valid, unless I missed something. What about keeping any tests we wrote in a collection?

[tmdoit-zz]

For me the point is to create API spec with examples and then based on it, generate collection - API spec first approach. Currently one missing feature is to generate from spec more than one example if response has more than one possible returned value per status. If I understand well, you are talking about collection first approach - generating API spec based on collection?

[saivan]

Well. I was going to use postman for my tests. So you generate the openapi file; then make a collection from it. After that, write tests in postman. It would be good to not have to lose all of your tests when you want to change the openapi file.

[preethammavin]

@saivan thanks for the feedback it is a valid use case, will keep it as a feature request

[tmdoit-zz]

@saivan now I get it - tests, not examples 🙂

[bikash8gupta]

The tests are deleted when a generate collection is applied. We definitely need a update collection because all collections have test suites in it and they are not kept when a new collection is generated. Till then API beta is not of any use. Why I will keep maintaining API specification as well as collection from two places.

[hilary]

What's the status of this issue? I would also like to use Postman API spec first. Without an 'Update Collection' option, the 'Generate Collection' option is useless.

Generated specs better than nothing, but nowhere near as good as authored specs. I really don't see the point of 'Generate Collection' as it stands. Perhaps you could explain a maintainable workflow that used 'Generate Collection' and did not use 'Update Collection'?

[sankalp0o]

@hilary @bikash8gupta @saivan thanks for reaching out!

Collections and specifications (any of them) are very different from each other and at this point, we're unable to update a collection from the spec which can reliably work 100% of the time. That being said, it's not completely impossible depending on what has changed in the spec and how much has the collection evolved. There are solutions that can work in some of the cases.

I request you to tell us the use cases that you have for updating the collection. What are you changing in the spec and how you imagine the collection to be updated? E.g. I'm adding new endpoints to the spec and I expect new requests being created inside the collection following the correct folder structure. E.g. I'm updating the descriptions for one of the endpoints and I expect the new description to replace the existing description in the collection request.

This will help us design a better solution! Please also let me know if you're open to getting on a quick call to understand your use case better.

[dvdvdmt]

I'm adding new endpoints or changing existing ones and I expect that different variable values such as baseUrl, authToken etc. will preserve in the collection settings.

[arlemi]

@sankalp0o Similar ask on our community forum: https://community.postman.com/t/automatically-sync-openapi-swagger-spec-to-collection/8551

Mostly, users are concerned about losing their tests and variables every time they regenerate the collection from the schema.

The only concern I have with importing the swagger spec each time is that our test and variables will not be maintained, unless I’m missing something. I believe even a 1 way sync that would update your Postman collection each time a change has been made to the swagger spec would be very useful.

I'll ask for more feedback if possible.

[andmoredev]

We use the Collection that gets generated by the API only for Documentation purposes so for my use case a full overwrite is completely acceptable. We keep our Test collections separately so that we can run tests for failures as well which would not be covered by an Open API definition file.

[hilary]

Even though I'm defining security in the spec, no auth is generated for the collection. That means I have to add auth to the collection after the fact. If generation were more complete, the lack of update wouldn't be as bad a problem.

We're primarily adding new endpoints, but occasionally deprecating existing ones. We're also updating documentation.

[tmdoit-zz]

Only difference between "Update Collection" and "Generate Collection" will be that "Update Collection" will recover tests written for defined collection endpoints. I use word "recover" because I don't see for this use case any reason to just not generate fresh collection from API spec and attach tests using collection endpoints as reference. I think that this way of solving problem will keep implementation of this feature simple.

---

UX: Use case: I want to update collection to keep written tests for collection endpoints

1. I click button "Update Collection"
2. In popup that appears I select via drop down list (API spec children) collection which I want to update
3. Postman updates collection preserving tests defined earlier for certain collection endpoints

Comments:

1. In popup windows information what collection update means

If I want to generate new, fresh collection without preserving tests I will use "Generate Collection" If i want to generate new, fresh collection with endpoints tests preserved I will use "Update Collection"

Preserving variables could be solved same or similar way I think.

This should make sense for API spec first approach.

[tank104]

+1 for this - especially when prototyping - its cumbersome. Right now I use Swagger, then when happy add it to Postman. Would be nice to skip Swagger step.

[fabiendeborde]

I'm adding new endpoints or changing existing ones and I expect that different variable values such as baseUrl, authToken etc. will preserve in the collection settings.

Pretty much this use case. If I update the OpenAPI file (which is the origin of my collection), I don't mind if my collection is overwritten but I expect that my collections variables, mock server URL and tests are preserved.

[darrencapner]

From watching the recent webinars and reading documentation, there appears to be a workflow in mind, but I find that there is a gap that I have not been able to find a solution for, and would be solved potentially with this issue.

My use case is if I am wanting to iterate on an API design through design, generate collections, deploy, get feedback, and modify design....how would Postman be used if there is no way to update existing collections where test scripts were added for contract testing and performance testing:

One such workflow details:

1. Make an API definition
2. Generate collections for contract testing, performance testing, documentation, working collection, etc.
3. Get feedback to iterate on that API design as new endpoints are added, and changes to existing API contract is changed
4. Generate new collections for contract testing, performance testing, documentation, working collection, etc. keeping all test scripts in those new version collections.

Questions:

1. Do we have to copy tests from existing collections to newly generated collections?
2. Is there a way to generate collections over-top of existing collections, only modifying request contract as per the changes in the API spec?

[a85]

Quick update here: this workflow is top of our minds.

We are looking at flows that allow you to:

1. Update a collection from an existing specification
2. Allow for pull request driven flows in case of conflicts between the new updated collection and the existing collection (so you can review and merge)
3. We are also looking at the reverse flow where you can update a collection and then optionally go backwards and update your specification (in the future, you will be able to do that with a pull request driven flow so you don't overwrite a schema right away)

Unfortunately, at the time being this is a manual process and honestly, we underestimated how popular this would be. Usage is exploding right now for the API tab and we are working hard to fix this loop. We just shipped an integration with GitHub and there is ongoing work with other repositories.

I'd like to thank everyone for their patience and sharing use cases with us. It helps us define the scope quickly and get things to production.

[dpkirchner]

My use case is vastly simpler: I'm writing openapi 3.0 YAML and then I click Generate Collection so I can preview the result to ensure it is correct. I'm not even to the point that I'm writing tests. There is a documentation link in the "Develop" tab and it takes you to a page that looks correct on the surface, however it is missing the API responses. If that page included responses my use case would be satisfied.

[arlemi]

@dpkirchner To display example responses on the documentation you need to add examples in your collection. You can learn how to do that on our Learning Center: https://learning.postman.com/docs/postman/collections/examples/

[cdolan92]

There are a lot of use cases with Tests, but I'd like to focus on Documentation for our use case. We have a Published Collection running on api.ourdomain.com, and would like to keep that updated easily as our developers improve our API.

Here is our ideal workflow:

1. Commit to a Github repository linked with an API in Postman, using the new integration @a85 mentioned above, when there are updated endpoints.
2. Sync that API in Postman, which imported our OpenAPI 3.0 spec, to a Postman Collection.
3. Easily manage the Documentation associated with that Collection - If new endpoints are added, they would presumably have no Documentation/markdown associated with them. If endpoints change, the pre-existing Markdown would persist the update.

Basically, I'm not seeing how I can easily update our documentation when our team makes updates to our API, without having a bunch of markdown files kept in source control, and then copy+pasting those into each endpoint in the API in Postman... a very manual task I want to avoid!

[ghost]

I'm surprised this functionality doesn't exist. API specs are the heart and soul of rapid prototyping. Having to recreate docs, mocks, tests, and other artifacts when the spec changes are the antithesis of the desired workflow. Man, I hope this gets fixed. Kinda a dealbreaker for using Postman during the API design phase :(

[arlemi]

Glad you're bringing this up @sylvester-rubidium ! 😀

We've started iterating on that feature and are looking for early feedback from users. Since this issue is full of users with good use-case for it we thought we'd ask here first. The session would be 45mins (over Zoom), starting with a short interview on how you use the API Builder and what you expect to achieve with it, followed by a usability test on a prototype. If this is something you'd like to take part in, you can book time with us next week using this Calendly link 📅, and if you have more questions please ask them here or directly to me on arlemi@postman.com.

Looking forward to hearing from you all. 🙂

[arlemi]

Hey everyone, just a quick message to let you know that we're still looking for feedback from you all on the "update collection" flow. We've got time slots today, tomorrow, or Thursday and you can book any available time using this link: https://calendly.com/arlemi/update-collection-ux-research?month=2020-05. 😃

[fabiendeborde]

I would love to participate but I have a deadline this week, and so much work left to do....

If you still need some feedback in June I'll be glad to participate!

[arlemi]

@FabienDeborde No worries, thanks for your interest anyway! We may ask for a second round of feedback in June once we have a working version of the prototype, will update everyone here if so.

[hilary]

@arlemi @a85 Just booked our time! Wanted to say how happy we are that Postman has prioritized this workflow!

[gustave001]

+1 for this

[obieo]

I'd be happy to discuss what our expected workflow would be if there are dates opened up for June.

[arlemi]

Hey everyone!

We've just released this feature in Postman 7.27, update your app (or grab the latest version here) and give it a try! You can find the full documentation in our Learning Center: https://learning.postman.com/docs/postman/design-and-develop-apis/validating-elements-against-schema/

If you have any additional feedback please add it here or on the Postman community forum. We're already tracking a feature request for being able to trigger a validation/update through an API in #8697.

[gustavovendramini]

Thanks guys, excellent feature.

Testing here, and for me, the new one's methods/actions added in API doesn't appear to include on updating the collection

[arlemi]

@gustavovendramini Thanks for the feedback! We're tracking what you've mentioned and are hoping to release it in the next version of the app. 🙂

[molteninjabob]

@arlemi, @a85 et al., I just found this thread trying to find a solution for a different issue I'm experiencing.

I've spent the last month building out an OpenAPI 3.0 schema for an existing api. The dev team already had a collection of requests (but no tests) and being able to validate the schema against an existing collection has been invaluable to ensure the schema I'm creating is correct. Great feature! I also just noticed the updated/improved visual diff, which is a nice addition to help troubleshooting.

I now find myself in an interesting position where I now have the schema created and I can generate a collection, but when I generate from the schema, all of the fields are set to the type that is expected instead of values that can be used, even if an example is provided.

This might be a separate issue, though I think still related, but there doesn't seem to be a way to generate populated variables from the examples in the schema. The examples created in the collection are great for mocking responses, but if I want to exercise the requests directly in postman, I have to go replace all query parameters, path vars and request body values in every request. As you can imagine, this is just as time consuming as copying/pasting tests would be.

I'd love to see Postman take the example values defined in the API schema automatically added as collection variables and inserted into the generated requests instead of just putting the type. If done correctly, the generated requests would be immediately usable and look like this:

[sankalp0o]

@molteninjabob thanks for bringing this up. We're putting in some configuration options so that you're able to select how you want your query params, request body etc to be generated. I'll post here what I have an update on this. If there's anything else that you'd like to see in the generate/update flows, do let us know!

[obieo]

Checking in on the "Update collection to retain tests" feature. I know this has been stated in multiple ways before but an ideal workflow for our team would be the following.

1. OpenAPI spec is generated upon our CI build and our repo or CI pipeline in Azure DevOps would be integrated with Postman.

2. Once integrated, the OpenAPI spec and Postman collection it's integrated with would be in sync each build to reflect changes in the spec. (New endpoints, deprecated endpoints, parameters, etc)

3. Test and variables are retain each time the OpenAPI spec updates the collection in Postman

This workflow would allow for a seamless source of truth where our team could truly collaborate on test, development and documentations.

Please let me know if something like this is being considered.

[arlemi]

@obieo You can already try out this feature, check out the documentation: https://learning.postman.com/docs/postman/design-and-develop-apis/validating-elements-against-schema/

Updating the generated collection would be a manual process for now but you'd still keep your tests etc. after every update. We're tracking a feature request to be able to do that flow using the Postman API in #8697.

[arlemi]

Closing this feature request as this has been out for some time now, we'll keep tracking improvements in other tickets to avoid confusion!

[obieo]

Hello – Thanks for the response however unless I’m missing something, this feature that is referenced does not hit on the scenario I was describing.

I see that you’re manually able to define the schema however what if I’m just looking to import the OpenAPI spec, using a url from the spec that I have already created a collection from.

When I go to import, I’m only left with 2 options (Copy, Replace). In this case I would not want to do either. I’d like to be able to import the same spec, pointing it to the existing collection generated from the spec, and update the schema automatically without deprecating the test that I’ve created for each test. I was over a url to some documentation but it does not cover being able to do what I just described.

Thanks,

Andrew Obaseki Crowe LLP Office: 317.706.2639 | Cell: 812.259.1829 andrew.obaseki@crowe.commailto:andrew.obaseki@crowe.com www.crowe.comhttp://www.crowe.com

From: Arlemi notifications@github.com Sent: Wednesday, August 12, 2020 5:22 AM To: postmanlabs/postman-app-support postman-app-support@noreply.github.com Cc: Obaseki, Andrew Andrew.Obaseki@crowe.com; Mention mention@noreply.github.com Subject: Re: [postmanlabs/postman-app-support] Add option "Update Collection" next to "Generate Collection" in APIs tab (#6722)

CAUTION: This email originated from outside of Crowe. Do not click links, open attachments or forward unless you recognize the sender and know the content is safe.

Closing this feature request as this has been out for some time now, we'll keep tracking improvements in other tickets to avoid confusion!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpostmanlabs%2Fpostman-app-support%2Fissues%2F6722%23issuecomment-672760230&data=02%7C01%7Candrew.obaseki%40crowe.com%7C344d48629f9143aa909008d83ea12a77%7C6ff60d36925f4785a854510f909ee561%7C0%7C0%7C637328209209969213&sdata=UUJTFVkLQlWj3RprQBqni44h9JB5IFTTPTYazsbMsPw%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAM3TFMAZZPTKKSENURLF4UTSAJNLNANCNFSM4HYUI2EQ&data=02%7C01%7Candrew.obaseki%40crowe.com%7C344d48629f9143aa909008d83ea12a77%7C6ff60d36925f4785a854510f909ee561%7C0%7C0%7C637328209209969213&sdata=%2BFjMpGv7OrMC1EIgE5L3YwPEhnAlHDqdvVjW0e0a1HU%3D&reserved=0.

This email message is from Crowe LLP or one of its subsidiaries and may contain privileged or confidential information or other information exempt from disclosure under applicable law. If you are not the intended recipient, please notify the sender by reply email immediately and delete this message without reading further or forwarding to others. This email is not intended to be a contract or other legally binding obligation, and any tax advice expressed in this email should not be construed as a formal tax opinion unless expressly stated. Visit www.crowe.com/disclosure for more information about Crowe LLP and its subsidiaries.

[tank104]

I agree with Obieo, I still don't see the feature to updte existing collection, unless I am mistaken.

[arlemi]

@obieo The current feature doesn't allow you to update a collection by importing a new version of the schema as a collection. It allows you to update collections generated from a schema in the API Builder. In your case you'd need to:

• Import your schema as an API
• Generate a collection from it
• Copy over your tests from your current collection to the generated collection

Once this is done you'll be able to update your schema in the API Builder and update your collection while keeping the tests. Please look at our documentation to see more details about that.

FYI when you import a schema and create a collection from it all that we're checking is if there's a collection with a similar name, we're not checking the content of the collection, therefore we wouldn't be able to offer to update a collection this way.

[obieo]

Thanks @arlemi for the explanation. We will try this in the meantime.

I do believe that a future enhancement of allowing users to update a collection based of changes to a schema, while maintaining the tests and variable would be very valuable. Especially as Postman continues to find was to automate the integration with OpenAPI.

An example of this workflow would be

1. Changes to API schema are made and committed
2. OpenAPI spec is updated from Azure DevOps build
3. Those changes would automatically sync to postman collection initially created in Postman (delta update)
4. Tests, variables, pre-request, etc would remain intact for that collection

[arlemi]

@obieo From my understanding what's missing here is the ability to have the update done automatically or triggered through an API. We're already tracking two similar requests around the same:

• Support OpenAPI-First, Forked Collection Workflow #8940
• Trigger an update to the collection from Postman API #8697

Give a 👍 on the original posts there if this is what you're looking for, this helps us prioritise! 🙂

[dvdvdmt]

@arlemi no it is not what @obieo described. Update of the Postman collection could be manual. In a simple form:

1. A user changes openapi.yml.
2. A user imports openapi.yml as a collection.
3. Existing collection updates but its tests, variables, pre-request remain intact.

[tank104]

@dvdvdmt is right - its just a manual button.
So in our case where we are prototyping/change frequently the api spec, we can update what we have already, without everything being replaced.

[arlemi]

@dvdvdmt @tank104 Understood, and this is the whole point of this feature. I think there's a misunderstanding as to how you're supposed to use it.

Postman allows API-first development and has a whole section of the app (called the API Builder) dedicated to authoring APIs as well as generating collection from a specific API schemas. You can create an API directly from the API tab or by selecting import as API instead of collection when importing a schema file. Once your API is imported, you can use the Generate Collection button to generate a collection, you can then either:

• link the API to a github repo to enable a 2-way sync between GitHub and Postman
• edit your schema directly in Postman
• copy/paste your schema file in Postman everytime there are changes

When this is done, you can check whether the collections previously generated need to be updated, and if so update them without losing tests, pre-request scrips etc.

You can see an explanation of the feature along with a demo on the API Builder webinar we ran a couple of months ago: https://youtu.be/Aolgf6tQaqQ?t=1378

[obieo]

@arlemi what you have posted makes sense however most of our teams aren't developing/authoring APIs within postman.

• They are doing so in their own IDE and committing those changes to the repo which then updates the OpenAPI spec upon the build.

• From there someone from the team (generally a tester) will go to Swagger, grab the spec url and import it into postman to generate the collection.

• At this point it would be nice to recognize the url and the associated collection and allow the user to update the collection based of the change in the spec.

Ultimately I agree with @dvdvdmt and @tank104 a good first step would just be a manual button to update the collection each time the spec is imported into Postman.

[cdolan92]

I agree with obieo - our team is pushing code out and we’re grabbing the Open API spec from the deployed code, which is then imported into Postman.

Once inside postman, updating the collection such that endpoint changes are accepted but documentation & markdown are preserved is the desired goal. This is particularly of interest because we’re using Postman’s hosted API documentation page.

The only work around to this problem is to create a new collection (C1) based off of updated OpenAPI docs, and clone my existing published Collection (C2, which is based off of C0).

Then, manually copy/paste endpoints from the newly generated Collection (C1) back into the copy of my original collection (C2), and publish collection C2 as my primary collection for our hosted documentation page, and delete C0 and C1.

---

From: obieo notifications@github.com Sent: Wednesday, August 19, 2020 9:36:51 AM To: postmanlabs/postman-app-support postman-app-support@noreply.github.com Cc: Charlie Dolan dolan.char@gmail.com; Comment comment@noreply.github.com Subject: Re: [postmanlabs/postman-app-support] Add option "Update Collection" next to "Generate Collection" in APIs tab (#6722)

@arlemihttps://github.com/arlemi what you have posted makes sense however most of our teams aren't developing/authoring APIs within postman.

• They are doing so in their own IDE and committing those changes to the repo which then updates the OpenAPI spec upon the build.

• From there someone from the team (generally a tester) will go to Swagger, grab the spec url and import it into postman to generate the collection.

• At this point it would be nice to recognize the url and the associated collection and allow the user to update the collection based of the change in the spec.

Ultimately I agree with @dvdvdmthttps://github.com/dvdvdmt and @tank104https://github.com/tank104 a good first step would just be a manual button to update the collection each time the spec is imported into Postman.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/postmanlabs/postman-app-support/issues/6722#issuecomment-676370247, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAQISJXXEBFDSHSSSTO4JALSBPIPHANCNFSM4HYUI2EQ.

[arlemi]

Ok, I think we're on the same page then!

In both your use-cases, your Open API spec is pushed somewhere, if that somewhere is a GitHub repository you can then have it automatically sync with Postman (no need to grab it anymore) and all that is left to do is update the attached collections (i.e. "a good first step").

If you're using a different git repository then go give a thumbs up to the feature request #8245. If you believe we should have a way to detect when an import is made from a similar URL then please open a new feature request and we can keep the discussion going there. This one was specifically about adding a way to update collections from the API Builder which is now feasible. 🙂

[obieo]

Thanks @arlemi. I have created feature request #8950 for updating collections on manual imports.

[pankaj-cashify]

@arlemi I do not see any option on UI to update attached collections for OpenAPI. Am I missing something?