Vycka / LoadRunner

Load-testing framework for writing load/stress test scenarios in c#
GNU General Public License v3.0
20 stars 5 forks source link

Does this works with asynchronous code, task async/await ? #12

Closed Ricky1997 closed 3 years ago

Vycka commented 4 years ago

Hello, sorry for late replay.

You can't use async await in IScenario.ExecuteScenario() directly:

For workaround: One can just use task.Result approach instead of await. E.g. HttpResponseMessage result = await httpclient.ExecuteAsync() vs HttpResponseMessage result = httpclient.ExecuteAsync().Result this approach is used here: https://github.com/Vycka/HttpMockSlim/blob/master/tests/HttpMockSlim.LoadTest/Scenario.cs

If your issue can't be workarounded like this, i would be curious to know more details about it.

Ricky1997 commented 4 years ago

Hi,

I want to know.that how is the tps value different from no. Of iterations/ time dimension.

How are total transactions calculated in one sec ?

Regards Ricky

On Wed, 22 Jul 2020, 06:20 Vytautas Klumbys, notifications@github.com wrote:

Hello, sorry for late replay.

You can't use async await in IScenario.ExecuteScenario() directly:

  • LoadRunner engine ultimately measures the duration it takes for IScenario.ExecuteScenario() method to complete.
  • But doing await inside void return methods (like void ExecuteScenario()) will cause method to complete instantly (as under the hood await registers the callback to the rest of the code block).

For workaround: One can just use task.Result approach instead of await. E.g. HttpResponseMessage result = await httpclient.ExecuteAsync() vs HttpResponseMessage result = httpclient.ExecuteAsync().Result

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Vycka/LoadRunner/issues/12#issuecomment-662448681, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHHQSIZCWPOZODCQNDAHTGLR43RR5ANCNFSM4OR2KFCQ .

Vycka commented 4 years ago

Hello, Transactions per Second metric it-self is straight forward. For iterations received it tracks: https://github.com/Vycka/LoadRunner/blob/master/src/Viki.LoadRunner.Engine/Aggregators/Metrics/TransactionsPerSecMetric.cs#L32

Metrics don't know anything about used dimensions in aggregation:

Iteration Id Iteration started T+s Iteration Ended T+s
0 0 1
1 1 1
2 1 2
3 3 5
4 4 5
5 5 6

If we only had KPI aggregation (no dimensions)

new HistogramAggregator()
    .Add(new TransactionsPerSecMetric());

All 6 iterations would be calculated into single TPS value: 6 / (6-0) = 1

But when we combine it with dimensions:

new HistogramAggregator()
    .Add(new TimeDimension(TimeSpan.FromSeconds(3)))
    .Add(new TransactionsPerSecMetric());

By default TimeDimension groups data by Iteration Ended value https://github.com/Vycka/LoadRunner/blob/master/src/Viki.LoadRunner.Engine/Aggregators/Dimensions/TimeDimension.cs#L15 And in this demo case, data gets broken into 3 groups:

Request Id Iteration started T+s Iteration Ended T+s
0 0 1
1 1 1
2 1 2

TPS = 3 / (2 - 0) = 1.5

Request Id Iteration started T+s Iteration Ended T+s
3 3 5
4 4 5

TPS = 2 / (5 - 3) = 1

Request Id Iteration started T+s Iteration Ended T+s
5 5 6

TPS = 1 / (6 - 5) = 1

As result of such math - TPS results are not 100% accurate, but difference is negligible if enough iterations are made within one group

Vycka commented 3 years ago

Given no additional comments, I assume that previous answer was sufficient.