Add support for pacing

[MattDodsonEnglish]

Feature Description

This is not just sleep. Pacing in this issue means a configurable, randomized min/max duration for each VU.

This came up in a discussion on the community forum. Basically, with pacing, users define a random iteration length between user-defined minimum and maximum values, effectively sleeping at the end of the iteration.

Users reported that such pacing allows for effective throughput control without needing to modify VU numbers, and mitigates the "coordinated omission" problem of the closed model, while keeping the benefits of having tests send data at a fixed rate across runs. The function would work something like this:

export default function(){
let startTime= new Date(Date.now());     //Timestamp start of iteration

  try {
    Application.LogIn();
    Application.Search();
    Application.AddToCart();
    Application.Checkout();
    Application.Logout();
  }
  catch anyCallFailure(){
    let endTime = new Date(Date.now()):   //Timestamp end of iteration
    let timeTaken = Math.abs(endTime-startTime); //Calculate the iteration duration
    sleep(randomIntBetween(minPaceTime, maxPaceTime)-(timeTaken/1000)); //sleep for a random amount of pacing time, less iteration duration
  }

let endTime = new Date(Date.now()):      //Timestamp end of iteration
let timeTaken = Math.abs(endTime-startTime); //Calculate the iteration duration
sleep(randomIntBetween(minPaceTime, maxPaceTime)-(timeTaken/1000)); //sleep for a random amount of pacing time, less iteration duration

}

But if a user can already do this, can't they write their own helper function? True, it's probably possible, but it might be worth to implement because:

• k6 aims to be a batteries-included load testing tool
• perhaps implementing this in Go could bring perfomance benefits.
• It would bring feature-parity with other load-testing tools, like NeoLoad, Gatling, and LoadRunner

Suggested Solution (optional)

Something like combining minimumIterationDuration with a random amount of sleep at the end of the test. It needs to be random to prevent patterns.

Already existing or connected issues / PRs (optional)

No response

[pablochacin]

Doesn't constant-arrival-rate executor achieve the same goal? https://k6.io/blog/how-to-generate-a-constant-request-rate-with-the-new-scenarios-api/

[MattDodsonEnglish]

It's similar, but the constant-rate executors both use the open model. If I'm not mistaken, the open-model effectively lets users avoid the "coordinated omission," The issue is that, in the closed model, as each VU starts only after the prior one finishes, the system might "coordinate" with the load generator, effectively slowing down its arrival rate,

But, despite the scary-sounding phrase, "coordinated ommission," there are many valid reasons to use the closed model―after all, that's why we offer 4 closed-model executors. Arrival rate is not the most important variable to control for all tests. For example, if you wanted to test your application has 100 concurrent users, I don't think you could control that in the open model. The same thing if you wanted to test how a SUT responds to fixed amount of data writes (easier to test with fixed iterations).

As I understand it, adding pacing provides a way to mitigate the tradeoffs of each system

[hailtonothing]

Doesn't constant-arrival-rate executor achieve the same goal? https://k6.io/blog/how-to-generate-a-constant-request-rate-with-the-new-scenarios-api/

No, it doesn't. The constant arrival rate executor is an "open model" executor that dynamically alters the number of VUs according to a transactional/volume target, and is not a repeatable load scenario for this reason. The addition (or reduction, for that matter) of VUs mid-test has a number of performance drawbacks for both load generators and SUTs. Increasing VUs to hit a target on a system that's underperforming to a transactional NFR, for example, is precisely the wrong response.

Pacing can have its own problems if implemented incorrectly, but this is down to the user's understanding of the system. Implemented correctly, it both avoids "open model" issues and mitigates issues associated with a "closed model", while controlling throughput as defined by the user.