search

artilleryio / artillery

The complete load testing platform. Everything you need for production-grade load tests. Serverless & distributed. Load test with Playwright. Load test HTTP APIs, GraphQL, WebSocket, and more. Use any Node.js module.
https://www.artillery.io
Mozilla Public License 2.0
8.04k stars 511 forks source link

VU continue running the 'loop' when assertion fails #684

Open vichybaby opened 5 years ago

vichybaby commented 5 years ago

Query 1: I am wondering if the VU could continue running the 'loop' when assertion fails e.g. the VU fails in assertion of round 15, then it goes to round #6

Query 2: Could I run the test in below patterns? e.g. Pattern 1: ramp up to 10 VUs in 1 minute, then loop for 1 hour Pattern 2: ramp up to 10 VUs in 1 minute, then loop for 100 times, then ramp down to 0 VU in 1 minute

hassy commented 5 years ago

What type of an assertion is that? a capture/ifTrue combo, a custom JS function or something else?

Wrt those patterns: arrival rates are independent of request loops, so the answer is sort of. You can ramp VU arrival up and down as needed, but there's no way to coordinate between creating new VUs and how many requests existing VUs may have sent or how long they've been running for.

We could extend phase specs to support a flag which would make Artillery wait for all VUs from the current phase to finish before starting the next one, which would work for Pattern 2 and sort of for Pattern 1. "Sort of" because you could use think to make VUs hang around for an hour, but request latency times would be added to the total running time of those VUs as well.

vichybaby commented 5 years ago

Thanks hassy! The assertion is a capture/expect combo. These are queries and I just want to know if they are doable.

As a new starter on ArtilleryIO, how can I can get full spec of ArtilleryIO? Sometimes I am confused about using arrivalRate or arrivalCount. For how it works in terms of rampUp/rampDown+scenario, could you add more explanations in document?

iunal commented 5 months ago

Query 1: I am wondering if the VU could continue running the 'loop' when assertion fails e.g. the VU fails in assertion of round 15, then it goes to round #6

Query 2: Could I run the test in below patterns? e.g. Pattern 1: ramp up to 10 VUs in 1 minute, then loop for 1 hour Pattern 2: ramp up to 10 VUs in 1 minute, then loop for 100 times, then ramp down to 0 VU in 1 minute

It has been long time since you asked, but I am wondering, did you find the way for Query1?