Phantom test disregards schedule duration

[Sirozha1337]

• Set a schedule with big RPS count and some duration, for example: schedule: const(100000, 10m)
• Start a server running slow operations on each request. Serving less requests than set in schedule.
• Start yandex-tank.
• Expected Result: Tank will make as much requests as it can and will stop in 10 minutes.
• Actual Result: Tank won't stop until it makes a number of requests equal to RPS * duration seconds.

It seems like a bug to me, but feel free to close it, if that's the desired behaviour.

[afilatov]

As far as I know it's the desired behaviour. Increased duration of the test with an open load (load_type: rps) usually means that at some point tank/phantom hit the limit of instances (defaults to 1000) and could not produce planned amount of requests per second. Additional test time is spent to send that "unsent" requests.

1) In order to solve the root cause of your problem you can increase the number of instances (instances: N) using the approximate Littles Law formula:

<Number of instances> = <Max RPS> * <Max response time, sec>

and use multiple generators if number of instances is too large (20+k).

2) In order to solve the symptoms (longer test duration) you can configure autostop criteria as follows:

autostop:
  enabled: true
  autostop:
  - limit(<desired test duration (e.g. 10m)>)

[Sirozha1337]

@afilatov, second solution sounds just like what I was thinking about. Seems like I somehow missed that part of the documentation or maybe it wasn't there a year ago. Anyways, I think this issue can be closed now. Thank you for your feedback!