search

danielpalme / IocPerformance

Performance comparison of .NET IoC containers
https://danielpalme.github.io/IocPerformance
Apache License 2.0
876 stars 157 forks source link

Incorrect Unity testing #87

Closed ENikS closed 6 years ago

ENikS commented 6 years ago

It seems you configure Unity with Interceptors for all of you tests. Interception introduces reflection into building strategy and is rather slow. It skews results dramatically. I would suggest either creating two adapters with and without interception or configuring Interception only if it is required by the tests.

Most of the containers do not have interception so it creates unfair advantage when compared to the Unity. To give objective performance comparison the playing field should be leveled.

As you can see in these tests your results are way off.

ENikS commented 6 years ago

As a side note I was wondering if you would be interested in migrating your benchmark to this platform? Disposable Application Domains do not provide as accurate results as the library I've linked.

ENikS commented 6 years ago

It looks like your adapter is missing implementation for Generics and ASP.NET as well. Let me know if you want help with implementing these.

danielpalme commented 6 years ago

Well, one could also argue, that Unity should take care of Interceptors and only enable it for classes that are configured with interception. But if you would like to change that, just do it.

I don't plan to update to Benchmarkdotnet, since it does not support multi threaded tests. We already discussed this in another issue (see #77).

If you like, you can add support for missing features like Generics and ASP.NET. Go ahead.

ENikS commented 6 years ago

Well, one could also argue, that Unity should take care of Interceptors and only enable it for classes that are configured with interception.

One would totally agree with you but unfortunately original Unity did not do that. Version 5.x is created as replacement for v4.0.1. and was kept compatible with the original.

Unity 6 will fix it.

danielpalme commented 6 years ago

But that is the problem of Unity, not the problem of the benchmark. If you change this now, we have to revert the change as soon as Unity 6 is available. It is nearly impossible to create a benchmark that treats all containers 100% fair.

ENikS commented 6 years ago

I have not decided yet if I want to support interception in Unity 6. Interception was not a good idea to begin with,

I submitted a pull request to fix the adapter