Log and analyze API traffic in dev env to aid development in many ways.

[andrewluetgers]

Is your feature request related to a problem? Please describe. Postman collections can get stale or never written in the first place. APIs have multiple users and they may be using the app in a way that is not documented in postman. I use postman to debug all the time, but I have to go to the logs to get the data to plug into postman to reproduce the error. I use postman all the time to aid development, to understand the data but I have to find the right data in the app to do the specific request I need to make. This slows things down and is prone to human error.

Describe the solution you'd like There is already a ton of factual information about my API and its usage flying around the network. I want to see all of these calls to an API collected and analyzed for the purpose of easing development. (This is not an ask for production monitoring as a service however that might be cool, this is more about aiding development.) Add a line of code to my app maybe a wrapper around the http lib or maybe it reads std out logs (I don't really care), maybe has some config, and it logs all requests and responses to the app.

Monitoring all this data provides opportunities to automatically document the real world usage of an API, this also shows how the system changes over time, and can be leveraged to automatically regression/snapshot test said api as well as auto-generate a postman collection. This data can be easily accessed to find error conditions and reproduce them for debugging, it can be tracked over time to automatically highlight changes to the api calls and response structure. It could be used to document latency.

Describe alternatives you've considered As far as I know there is nothing that really does this. I have some tools integrated into our app for snapshot testing would could be used more pervasively and then analyzed as I am describing but I don't have the time to develop that.

[prashantagarwal]

@andrewluetgers That's an interesting use case. I believe you are on the right track in terms of a viable solution as well. Most of the things that you can do with the Postman app can also be done using Postman APIs & you can use this build a quick solution. I'll try to summarize my brainwave below:

• Identify the requests that you are interested in. Maybe you are only interested in the requests that resulted in some error.
• Create a collection in your Postman app.
• Write a utility in your server that will fetch this collection using Postman API and then update this collection by adding this request to that collection.
• If there is a large number of requests that you want to sync to your app then I would recommend creating these collections dynamically as well (Maybe a collection for every date.)
• You can similarly write a script to scrap your logs for the requests that you are interested in and Sync those to your account.

I hope this should solve your use case but feel free to reach out in case you need more details.

[andrewluetgers]

Thanks, did not know about the Postman APIs. Honestly I am unlikely to have time to implement this. Just surfacing the request because I think it would be a good way to automate a lot of things that are currently manual in my dev/debug workflow.

[sankalp0o]

@andrewluetgers I agree this can be very useful! We don't have any plans to tackle this right away but we definitely are thinking in the same direction. Will let you know when we are planning for something like this. In the meantime, maybe reach out to the https://community.getpostman.com/ and see what people are doing. I'd also love to get more inputs from others if they're looking for a similar solution.

[jotsarup]

We are working on the use case you mentioned. We will be analysing log data (API access logs from AWS API Gateway for first cut) to fetch requests and associated data. The collected requests can then be used to create/update collections. The data for the requests that caused an error can then also be analysed from within Postman.

But we are currently integrating with AWS API Gateway. In future we plan to integrate other Gateway and APMs from where API Access logs can be fetched.

@andrewluetgers Would love to understand your use-case in detail? Would you be using a gateway in testing or are using a local environment.

[andrewluetgers]

I would not have thought of the API Gateway integration thats pretty interesting tho hopefully it is not a pain to set up.

I could go on and on with this subject as I think this is a very rich area for dev tooling and is pretty much untouched. There is a simple and a dream version. The simple is just watch a server's traffic incoming and what it responds with. You can overlay all kinds of smarts to discover across multiple requests the schema of inputs and responses and produce postman api documentation automatically from that.

You could show things like here is an endpoint here are all the requests to that endpoint, all the post bodies and all the json responses. if you had that over time for all the endpoints you could have monitoring messages, you could do automated testing by just replaying things, if the shape of the data changes for example, thats probably expected in development but I may not know what those changes all are. You could diff the shape of the response from today to last week.

These recordings if they come from dev environment may warrant treatment as just a log of activity, if from a qa environment they may warrant monitoring to quantify how many and what aspects of the api endpoints were touched during the qa session. We can then mark request-response pairs as valid and use them to automate testing.

So what this does is it helps me out during development by making my service layer request-response activity more readable, interactive and replay-able via all the postman goodness without having to manually enter it in.

This can lead to free documentation, in theory.

This can also lead to better qa and automated testing for free, in theory.

I'm sure you can imagine all kinds of other things to do with this stream of data.

[andrewluetgers]

@jotsarup I just realized I did not really answer your question, our app goes something like browser -> AWS ALB -> Nginx -> Node Express App -> multiple services So this would need to sit in front of the node app or be integrated as middle-ware or intercept traffic in some other way.

[shashankawasthi88]

@andrewluetgers thanks for the reply, while we are trying to work out the exact solution to your use can, can you confirm if something like a Postman Proxy/ Interceptor might help fulfill your use case? Both proxy and interceptor can help you capture the HTTP traffic into Postman. Let us know your thoughts, while we work towards finding the most optimized solution.

[andrewluetgers]

@shashankawasthi88 wow this is awesome, I'm excited to put these to use today!

[shashankawasthi88]

@andrewluetgers were you able to use Interceptor/ Proxy to solve some of your use cases? We are still looking into a more cohesive solution to address your complete use case end to end, where we could fetch the API data/ logs from your environment and pipe that into Postman (in the API object).