Thorughput time to execute accuracy wildly incorrect

petabridge / NBench

Performance benchmarking and testing framework for .NET applications :chart_with_upwards_trend:

https://nbench.io/

Apache License 2.0

532 stars 46 forks source link

Thorughput time to execute accuracy wildly incorrect #139

Closed jamesbascle closed 4 years ago

jamesbascle commented 7 years ago

I have a benchmark with this setting

    [PerfBenchmark(Description = "Test performance of Matching.", RunMode = RunMode.Throughput, RunTimeMilliseconds = 200, TestMode = TestMode.Test)]

It runs on average between 8-10 seconds.

You can see the test here: https://github.com/jamesbascle/OneOf/blob/DiscU/OneOf.Tests/MatcherPerformanceTests.cs#L10

alexpantyukhin commented 7 years ago

Maybe setting NumberOfIterations parameter will work for you (for example 3). The default value is 10 for NumberOfIterations (https://github.com/petabridge/NBench/blob/126e1d9c832f57039a2f598ab47003216f22c728/src/NBench/PerformanceBenchmark.cs#L48) .

jamesbascle commented 7 years ago

Ahhhh. I see. I understood the documentation to mean that one used Throughput AND RunTimeMilliseconds OR Iterations AND NumberOfIterations, not that those are used in conjunction.

Aaronontheweb commented 7 years ago

@jamesbascle sorry about that - we should rename this issue to clarify that in the documentation then I think. Isn't that the real issue here?