Open CherryDT opened 4 years ago
I had issues with Artillery's rampTo being inconsistent as well during repeated runs. After seeing your issue I got curious and looked in the code.
It seems they use some random probabilities to determine if an arrival happens within a given period for rampTo. I read the code rather quickly, but it would explain the inconsistent results.
let probabilities = crypto.randomBytes(Math.ceil(spec.duration * 1000 / tick * 1.25));
// (...)
let prob = probabilities[i++] / 256;
if (prob <= p) {
ee.emit('arrival', spec);
}
(Core/lib/phases.js createRamp)
They did -not- document this correctly iirc and it makes repeated test runs possibly incomparable. I had a much better time just manually writing out the phases myself, at least it was predictable.
Is this still the case? We're evaluating artillery amongst other options and this kind of thing is a dealbreaker for us.
Note: I've previously asked this on Spectrum but got no response for 6 weeks, so I'm escalating this to a GitHub issue because it smells like a bug. If I'm just doing something wrong, let me know.
Hi, I have issues using
rampTo
. From what I understand, a step withduration: A
,arrivalRate: B
,rampTo: C
is supposed to linearly ramp the rate fromB
toC
over the course ofA
seconds, right?However, I consistently get a different behavior which makes it hard to run the tests properly. The ramping step takes several times longer than specified (for example 4 minutes when
duration
was set to 60) and ramps only to a fraction of the defined rate (for example 30%) only to have artillery then state that the step is supposedly "complete".Here is an example of my issue:
Config:
I'd expect this to ramp from 100 req/s to 2000 req/s over the course of 20 seconds and then continue at 2000 req/s for another 20 seconds.
What happens instead, though, is this: It ramps from 100 req/s to 843 req/s (!) over the course of 60 seconds (!) and then jumps to ~2000 req/s for another 20 seconds. I remove the
rampTo
phase and start with 2000 req/s right away, it works as expected.Here an excerpt from the log of this test, showing the issue:
What am I doing wrong?
Thank you!
(This is on 1.6.0)