Migrate performance benchmarks in unit tests to benchmarkdotnet

AdamsLair / duality

a 2D Game Development Framework

https://adamslair.github.io/duality

MIT License

1.4k stars 290 forks source link

Migrate performance benchmarks in unit tests to benchmarkdotnet #843

Closed Barsonax closed 4 years ago

Barsonax commented 4 years ago

Moved all our benchmarks in our unit tests to a separate benchmarkdotnet project.

How to use:

cd to Benchmarks\DualityBenchmarks
Run dotnet run -c Release --list tree to view the available benchmarks
Run dotnet run -c Release --filter CloneProviderBenchmarks to run the CloneProviderBenchmarks

TODO:

Some benchmarks are running alot of itterations. This is a leftover of the setup in the unit tests and is usually not needed unless you are interested in how something scales. Change this to only run once or if scaling is important parameterize it so its easier to see the scaling (for example do 3 runs with the first run only 1 itteration, the second one 10 itterations etc). DONE

Rebased on https://github.com/AdamsLair/duality/pull/842

ilexp commented 4 years ago

Should probably not merge this until #842 is addressed due to the rebase, reviews should be easier afterwards as well.

Barsonax commented 4 years ago

As #842 has been merged this can now be reviewed.

Before putting all current performance tests in the new project iam curious what do you think about this new setup? @ilexp @SirePi

ilexp commented 4 years ago

Overall setup looks pretty good 👍

However, this doesn't seem right:

devenv_2qweZIQbRw

Looks like a missing ignore condition in the .csproj?

Also, I'd consider copy-pasting TestObject next to the benchmark to avoid the dependency to the DualityTests project, since I'm not sure if that should be there. Not a change request though, just food for thought.

Barsonax commented 4 years ago

Overall setup looks pretty good 👍

However, this doesn't seem right:

Looks like a missing ignore condition in the .csproj?

Probably, haven't spend much attention to polishing everything as I was mostly interested in how you find the benchmarkdotnet way of doing things.

Also, I'd consider copy-pasting TestObject next to the benchmark to avoid the dependency to the DualityTests project, since I'm not sure if that should be there. Not a change request though, just food for thought.

Ye when we move all the benchmarks it should make it more clear we don't need the dependency. Though in benchmarks you might wanna reuse the same models as in the tests (atleast I did in Singularity so much that I moved those to a separate project even).