I don't quite understand the method 'WithScenarioCompletionTimeout'

PragmaticFlow / NBomber

Modern and flexible load testing framework for Pull and Push scenarios, designed to test any system regardless a protocol (HTTP/WebSockets/AMQP etc) or a semantic model (Pull/Push).

https://nbomber.com

Other

2.08k stars 129 forks source link

I don't quite understand the method 'WithScenarioCompletionTimeout' #745

Open Libra321 opened 2 months ago

Libra321 commented 2 months ago

My NBOMBER version is V5.7.0,Refer to your timeout demo,I have changed the timeout to 10, 4, and 3 respectively,Unable to understand the results

AntyaDev commented 2 months ago

Hi @Libra321

After the Scenario duration (1 sec in your case), NBomber does some preparation and calculation, which takes some time. After this, it will run internal function waitOnWorkingActors() and wait on completion of all scheduled Scenarios (ScenarioCompleationTimeout).

So basically you have: Scenario duration (1 sec) + NBomber finalization + 3 sec (ScenarioCompleationTimeout)

AntyaDev commented 2 months ago

@Libra321 Do I understand that you want NBomber to stop measuring requests at the exact end time of the scenario?

Libra321 commented 1 month ago

@AntyaDev Yes, otherwise the test results would be unbelievable

AntyaDev commented 1 month ago

Got you. One more question: what are your thoughts about the following use case:

You defined a Scenario with a duration of 30 sec.
Let's imagine that handling one request takes 1 second. What should happen with a request that started at 29.5 seconds? This request will be finished in 30.5 seconds. Do you want to see it in the NBomber stats report, or should it be ignored?

Libra321 commented 1 month ago

@AntyaDev Normally, if the task is not completed within 30 seconds, it should be ignored. However, if the "WithScenarioCompletionTimeout" time is set, it should be effective

AntyaDev commented 1 month ago

Ok, let me think what we can do about it. I will ping you a bit later.