Request a way for nodes in collection to be logic-only, no request issued

[JohnArrowwood]

See also #2749, which was closed without understanding what was being asked for

In order to create collections that implement logic, I create nodes as branch points for loops and flow control. For example:

• INITIALIZE LOOP VARIABLES
• FOR EACH XYZ
• | First Request
• | Second Request
• NEXT XYZ
• LAST XYZ

In a collection like this, the only thing the ALL-CAPS requests do is manipulate global variables and control the flow of execution. For example:

• INITIALIZE LOOP VARIABLES - sets the loop control variable to its initial state
• FOR EACH XYZ - tests if the loop control variable is outside of the acceptable range, and if so, jumps to LAST XYZ
• NEXT XYZ - iterates the loop control variable, and jumps to either "FOR EACH XYZ" or "LAST XYZ"
• LAST XYZ - does nothing, it's only there as a branch point

I also sometimes also insert "requests" that are purely cosmetic, or simply toggle global variables around.

None of these "requests" actually need to make a request But they are needed, given the way that flow control was implemented.

What is needed is a way for those "requests" to not actually make any requests.

As a workaround, I simply point them at www.example.com. But that's not fair to them, my collections all generate traffic to their website needlessly. And it slows down the execution of my collection to wait for their website to respond.

[shamasis]

@JohnArrowwood this use case and many that I see (and personally face) each day are extremely interesting and shows how we are composing APIs to get ever-increasing value out of development.

As such, there are a few things we are thinking along those lines, namely - trying to figure out a way to do visual composition of APIs using a new DSL for the API-first development ecosystem. There should simply be a way to provide request definitions and then link or compose them using machines that do conditions, loops, map, reduce, etc.

I hear you and we have just started doing some proof of concepts and brainstorming to lay out a public RFC along those lines. The biggest hurdle is that every underlying aspects of Postman expects a valid request and also a valid response. An example you can see with how test scripts are not executed if a request is malformed or a response times out. These are not straightforward to change without breaking millions of existing collections.

A good first step would be to introduce an execution mode that uses this new Workflow DSL to compose requests and code the behaviour in such a way that the new runner (in this mode) would behave like you mentioned, i.e. proceed naturally for blank / malformed request and optionally allow users to handle the behaviour within the request composition definition.

What do you think?

[JohnArrowwood]

I think I'd like to know more about this DSL. :)

On Wed, Jan 2, 2019 at 1:44 PM Shamasis Bhattacharya < notifications@github.com> wrote:

@JohnArrowwood https://github.com/JohnArrowwood this use case and many that I see (and personally face) each day are extremely interesting and shows how we are composing APIs to get ever-increasing value out of development.

As such, there are a few things we are thinking along those lines, namely

• trying to figure out a way to do visual composition of APIs using a new DSL for the API-first development ecosystem. There should simply be a way to provide request definitions and then link or compose them using machines that do conditions, loops, map, reduce, etc.

I hear you and we have just started doing some proof of concepts and brainstorming to lay out a public RFC along those lines. The biggest hurdle is that every underlying aspects of Postman expects a valid request and also a valid response. An example you can see with how test scripts are not executed if a request is malformed or a response times out. These are not straightforward to change without breaking millions of existing collections.

A good first step would be to introduce an execution mode that uses this new Workflow DSL to compose requests and code the behaviour in such a way that the new runner (in this mode) would behave like you mentioned, i.e. proceed naturally for blank / malformed request and optionally allow users to handle the behaviour within the request composition definition.

What do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/postmanlabs/postman-app-support/issues/5707#issuecomment-450947474, or mute the thread https://github.com/notifications/unsubscribe-auth/AHd1xcgBkVmqg3Yi8IGaOxz6vrinn3Jdks5u_P4PgaJpZM4Zm8bj .

-- John Arrowwood

[albertotn]

@JohnArrowwood do you think something like https://en.wikipedia.org/wiki/Business_Process_Model_and_Notation ?

[JohnArrowwood]

First response:

It's great for novices, but probably a poor solution for power users, unless implemented skillfully.

Compare JMeter and Gatling. JMeter uses a visual, node-based model. Simple things are enabled by the nodes, but it gets increasingly difficult to do complex things. Gatling, by comparison, is implemented using a DSL written in scala. As a result, you have the full power of an advanced programming language to let you build whatever you need to build.

If you do something similar, a DSL that can be interacted with programmatically, yet can be interpreted to produce those diagrams, then you get the best of both worlds. Novice users can build the diagrams, and the digram is converted to code. Advanced users learn how to use the code, and then apply programming practices to optimize and build upon it.

As someone who has used both JMeter and Gatling, I have a very strong preference for the programmatic approach that Gatling affords. If I were only doing simple things, I might not have cared, or seen the benefits.

With the complexity of the things I do using Postman, I would probably abandon it as a tool if I were to lose the ability to programmatically compose workflows. Whereas, I would use it even more than I do if it were a more efficient way of tying together a workflow from a pool of pre-defined requests.

What I'd like to see, in a perfect world:

I have one folder per endpoint, and one request per method on that endpoint. Then, I can create a "workflow" that lets me call those requests by reference, possibly even adding custom additional pre-request/post-request code to be executed to extend the behavior provided by that saved request.

Make workflows reusable. In a workflow, I can call another workflow, the same way I do a request. And workflows can be organized in folders. A folder's contents can be a workflow, or a single element within the folder can be the whole workflow. It all depends on which item you choose to "run"

Make workflows in either GUI or CODE. Behind the scenes, it's all code.

Hope that helps.

On Sat, Jan 5, 2019 at 3:59 AM albertotn notifications@github.com wrote:

@JohnArrowwood https://github.com/JohnArrowwood do you think something like https://en.wikipedia.org/wiki/Business_Process_Model_and_Notation ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/postmanlabs/postman-app-support/issues/5707#issuecomment-451639415, or mute the thread https://github.com/notifications/unsubscribe-auth/AHd1xTAz7HG77iFf7Yv--vJ_V7mpXzm2ks5vAGlfgaJpZM4Zm8bj .

-- John Arrowwood

[albertotn]

@JohnArrowwood thanks for let me know of Gatling! Postman is defined as "Api development platform" and Gatling "performance testing". For me is something that can work together.. but are not the same. I'm also intrigued by some sort of high level DSL, because draw diagrams is a mess if there is a "big scenario" to model. I wonder also if, using something like this high level DSL, is possible to analyze and derive new api needs. For example: if in my tests I call always service1 then service 2, then 3, maybe this could hide into a new endpoint 4, just wondering!

[JohnArrowwood]

Correct, Postman and Gatling are not competing tools.

But JMeter and Gatling are. The point in mentioning them is that what you have proposed takes you down a similar path that JMeter took. I'm recommending that you go down the path that Gatling took. Instead of trying to shoe-horn everything into a GUI, create a concise, usable DSL that lets you express exactly what you need to do. Build a nice GUI on top of that, sure, but do not rob smart people of the ability to employ their programming abilities. Layering a nice DSL on top of a full-featured programming language is definitely the way to go, in my (maybe not so humble) opinion. :)

On Tue, Jan 8, 2019 at 7:28 AM albertotn notifications@github.com wrote:

@JohnArrowwood https://github.com/JohnArrowwood thanks for let me know of Gatling! Postman is defined as "Api development platform" and Gatling "performance testing". For me is something that can work together.. but are not the same. I'm also intrigued by some sort of high level DSL, because draw diagrams is a mess if there is a "big scenario" to model. I wonder also if, using something like this high level DSL, is possible to analyze and derive new api needs. For example: if in my tests I call always service1 then service 2, then 3, maybe this could hide into a new endpoint 4, just wondering!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/postmanlabs/postman-app-support/issues/5707#issuecomment-452281200, or mute the thread https://github.com/notifications/unsubscribe-auth/AHd1xZ__ru1SNHJJexW40IP12WSbrlxBks5vBI7egaJpZM4Zm8bj .

-- John Arrowwood

[a85]

@JohnArrowwood Agreed. That's exactly we are heading. This was also the underlying reason why we built a Javascript editor in Postman first instead of trying to do everything through a UI. Javascript will probably continue to be the underlying language for the DSL. The UI that we want to build will be on top of the DSL. We see that there is massive friction for people just getting introduced to Postman when they want to do simple use cases of chaining and testing. We want to simplify those use cases without compromising on the power of the product.

That said, we do have certain ideas on how the GUI can be done properly. Two ideas that I have been thinking about:

1. Composition: How to put together building blocks in a flow and then use the whole thing as a unit so the representation feels less complex.
2. Data flow: Illustrating data flows through a program and more controls to change it on the fly so people can play around in real-time how things will change with different parameters.

[albertotn]

@a85 cool a nice DSL is a plus, but decide what is a "nice" DSL is the hard part for sure. From user perspective it would be great if Gatling and Postman share same DSL (some kind of "standard" involved here?) so I can switch or reason on same language, I'm asking too much I know it!

About gui: data flow, do you mean somthing like Pentaho (see only images)

[JohnArrowwood]

@albertotn - the only way Gatling and Postman can share the same DSL is if Postman were completely re-written from scratch using Scala, and it compiled and ran Scala code for its scripting language. That's asking an awful lot. Sure, it would be cool if you could use the same code for both, but even as a Gatling enthusiast, I think it's asking too much. The problem domains are too different.

More achievable is to learn from how Gatling implemented its DSL, and create something similar. Or better yet, look at all the available web request DSL's out there, and synthesize something even better.

As I'm saying that, here's what is going through my mind:

Create a "class" that represents a single request. What we call a "request" now translates directly into code that instantiates and appropriately initializes an instance of that class. Within code, you can use that same class to manually create requests. Now, you have a path to go from GUI to code.

Where things get tricky is creating serial request execution. Users are going to expect that when they say "Execute this request" that the code that follows will only be executed once the request is complete. And that is a reasonable expectation, one that should be possible to meet. JavaScript makes that difficult. I'm not even aware of a way of doing so in a generic way. The JavaScript model is to do things using Promises or Callbacks, which is powerful but very hard for a novice to wrap their heads around. I do NOT recommend that Postman build its DSL around the same model.

Gatling works around this by having the DSL construct a data structure which gets executed, step-by-step. Unfortunately, this creates a situation that often trips up new users. They don't understand the difference between build-time and run-time execution. I would avoid duplicating that pattern, if at all possible.

The only thing that is left is some way of making JavaScript wait for an asynchronous process to finish before returning from a function. And that won't work, because then the GUI would stop responding while you wait for the code to finish executing.

The only other option that I can think of is for the Scripting Language to be something other than JavaScript. I imagine that option is dismissed without any serious contemplation, because it is way too big of a change.

What is left is standing between a rock and a hard place. The laudable goal of building a simple, easy, novice-friendly solution for building a series of requests in code becomes next to impossible without some serious re-thinking.

As much as I want to see it done, I don't have a ready solution for how to do it. If something comes to mind, I'll let you know.

On Thu, Jan 10, 2019 at 3:16 AM albertotn notifications@github.com wrote:

@a85 https://github.com/a85 cool a nice DSL is a plus, but decide what is a "nice" DSL is the hard part for sure. From user perspective it would be great if Gatling and Postman share same DSL (some kind of "standard" involved here?) so I can switch or reason on same language, I'm asking too much I know it!

About gui: data flow, do you mean somthing like Pentaho https://www.hitachivantara.com/en-us/products/big-data-integration-analytics/pentaho-data-integration.html (see only images)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/postmanlabs/postman-app-support/issues/5707#issuecomment-453007450, or mute the thread https://github.com/notifications/unsubscribe-auth/AHd1xdn8xEE4DJyy9Q0JaVUcgi39BMTtks5vBvbQgaJpZM4Zm8bj .

-- John Arrowwood

[jhardin-aptos]

Somewhat related to this:

It would be nice if requests and folders could be interleaved, rather than all folders appearing before all non-foldered requests. That would allow better management of a large looping test collection. Using @JohnArrowwood's original request as a basis:

• INITIALIZE LOOP VARIABLES ← JS-only request
• FOR EACH XYZ ← JS-only request
• 📁 First Request ← folder of related normal requests, e.g. a POST/GET/DELETE sequence testing one behavior
• 📁 Second Request ← another folder of related normal requests
• NEXT XYZ ← JS-only request
• LAST XYZ ← JS-only request

[TenguTech]

I am also interested in the addition of a 'logic only' node, or a way to skip the execution of the current API call in the pre-request script.

My Scenario The use-case I have for this is to deal with the limitation of the reading in data from CSV/JSON file in the collection runner. I need to read all rows from a CSV and send them in one call.

To achieve this I have one node that concatenates through the iterations. And a second node that is only run on the last iteration to send the collected payload.

The problem is the first node needs to make a valid call for each iteration. And all the unneeded call it makes, especially when the CSV is quite long.

Suggestions I can see that @shamasis has suggested adding a 'pm.break' in a number of issues.

Something to stop the current node in the pre-request script, or a node that doesn't make requests would be very useful.

Related Issues There are a number of Issue I can see that relate to this request:

Skip current node:

• Early quit the test script if test fails #1757
• Skip First Request #3197
• Ability to skip endpoint on the pre-request script #1172

Skip all nodes of an iteration, where setNextRequest() can't skip the first:

• Conditional logic in test to skip rest of iteration's API calls, move on to next iteration #3446

From the community:

• setNextRequest in Pre-Request - Skipping Call?

[JohnArrowwood]

I find myself generating folders full of nodes for the sole purpose of setting up global variables to use in other request flows. And every one of them is configured to hit http://www.example.com/. I don't really NEED to establish a connection or download any HTML, since all I'm doing is setting variables...

I could really use this feature. Any update on when we might have something?