search

danielpalme / IocPerformance

Performance comparison of .NET IoC containers
https://danielpalme.github.io/IocPerformance
Apache License 2.0
874 stars 157 forks source link

Multithreaded performance is not being measured accurately #110

Closed Barsonax closed 5 years ago

Barsonax commented 5 years ago

Noticed that the current benchmark does not measure the multithreaded performance accurately enough, especially when containers like singularity can do their work in less than 10 nanoseconds. I believe this is because there's too much overhead in the way the benchmark is done currently. Increasing the sample size might lead to more accurate results.

When you do this you will see that for alot of containers the multithreaded results will be faster like they should be.

danielpalme commented 5 years ago

Correct, but then the execution of the complete benchmark would take several hours. Not every container is as fast as singularity.

Barsonax commented 5 years ago

Good point.

A solution could be to execute a variable amount of itterations per adapter. Benchmarkdotnet does something like this by calculating how much itterations fit in a certain time frame.

While calculating that might be a bit complex simply giving a adapter a way to specify how many itterations should be done should work as well.

danielpalme commented 5 years ago

If you like you can change that. I won't invest any time in this benchmark any more (apart from updating the containers from time to time).