ComputationalRadiationPhysics / haseongpu

HASEonGPU: High performance Amplified Spontaneous Emission on GPU
http://www.hzdr.de/crp
Other
7 stars 6 forks source link

Benchmarking (and testing) framework #53

Open slizzered opened 9 years ago

slizzered commented 9 years ago

This could be split into 2 issues, but for development of ideas 1 issue might be sufficient now... both features will be closely related in code. (after doing a test, just record the time...)

It would be great to have some code that will a) test some defined input/output of the whole application (we had this before, but it was never really easy to see immediately if the results are the same or minimally different). Might be difficult since results are not 100% deterministic b) maybe also some unit-tests for smaller functions (mostly necessary if later some internal stuff will be improved/changed, like #51) c) create some fixed benchmarking scenarios (for whole simulation and also for single functions), so we have a fixed and reliable time value that we can use to measure performance changes that are introduced through new features. It would be good to have quite a list of parameter combinations. d) We should have a PERFORMANCE.md file that holds those values for different releases, so we can track the speed development over a longer timeframe

Things to compare in the future:

slizzered commented 9 years ago

10 and #54 will be helpful to support the implementation of this issue

erikzenker commented 9 years ago

I had a look into some testing frameworks and finally decided to use Boost.Test. There were no big differences between the test frameworks, but since we are using boost anyways there is no reason to ship more dependencies with our code.

slizzered commented 9 years ago

very reasonable thinking

erikzenker commented 9 years ago

Just to give a reference to ongoing work of this issue.

erikzenker commented 9 years ago

We should get rid of a matlab based testing proceedure. Better would be some free available scripting language!