Zooz / predator

A powerful open-source platform for load testing APIs.
https://zooz.github.io/predator/
Apache License 2.0
574 stars 109 forks source link

Fail test if request in Before flow fails #201

Open NivLipetz opened 5 years ago

NivLipetz commented 5 years ago

Describe the solution you'd like If a request in the Before flow fails, I would like the test to fail. This can be done by adding for each request in the before flow an optional parameter such as allow_failure: false while the default value would be true. If a request with the allow_failure: false fails, then fail the test; otherwise proceed.

enudler commented 5 years ago

How do you define failed request? By status code? Capture? Assert?

NivLipetz commented 5 years ago

Good point. I suggest we use artillery-plugin-expect documented here: https://artillery.io/docs/plugin-expectations-assertions/

image

Our challenge is to decide whether to PR to artillery or to wrap the before flow and implement this functionality in this project

enudler commented 5 years ago

Sounds good or we can use asserts which are supported out of the box in artillery I will add it to 1.2.0

enudler commented 4 years ago

It can be also implemented using the processors feature:

For example: function throwErrorOnError(requestParams, response, context, ee, next) { if (response.statusCode !== 200 || response.statusCode !== 201 || response.statusCode !== 204){ throw new Error('Stopping test'); } }

We will need to catch this errors and notify predator that test was aborted. Adding it to 1.3.0

enudler commented 4 years ago

I suggest aborting the test if a capture fails in the 'before' section Change need to be done in predator-runner