Performance measurement

[GoogleCodeExporter]

Hi,

i noticed that you're often using very quick benchmarks to measure
performance. Even though your results all seem to be correct i suggest to
use a slightly more advanced pattern to improve the credibility of your
tests. I think a test run lasting less than a few seconds is too inaccurate.

Usually every test run should take at least one minute. Additionally, there
should be at least 3 test runs, whereas the first test run is to be ignored
to make sure the machine is fully present at the second run. The following
runs are averaged.

I'm not yet very familiar with the .NET platform, but i guess using this
method even very doubtful critics can be convinced of your results. I'm
used to perform the above procedure in my Java benchmarks and it works great.

Hope this is the right place to bring up this issue. And thank you for DNP,
the site was helpful many times to me. :)

Kind regards,
mafutrct.

Original issue reported on code.google.com by mafutr...@googlemail.com on 5 Dec 2008 at 7:46

[GoogleCodeExporter]

It has been suggested I use Stopwatch also.

Your suggestion is good that I use longer runs and average them. For my biggest 
article I did about 12 hours of benchmarks:

http://dotnetperls.com/Content/Browser-Memory.aspx

but that is different.

I would like to see your Java benchmarks if you have them online. Just out of 
curiosity--I don't do Java :)

Original comment by allen....@gmail.com on 6 Dec 2008 at 12:43

[GoogleCodeExporter]

I'm sorry, i don't have any files online. The benchmarks are kind of easy to 
write
and use, though. If you're interested in this topic, i recommend to have a look 
at
JSN (http://www.javaspecialists.eu/archive/archive.jsp). Even though it is 
specific
to Java, most of their principles also apply to C# and any other modern 
language and
have always been a great source for digging into fun stuff.

Kind regards,
mafutrct.

Original comment by mafutr...@googlemail.com on 13 Dec 2008 at 11:10