search

FLAMEGPU / FLAMEGPU2

FLAME GPU 2 is a GPU accelerated agent based modelling framework for CUDA C++ and Python
https://flamegpu.com
MIT License
107 stars 22 forks source link

Performance Regression Testing #275

Open ptheywood opened 4 years ago

ptheywood commented 4 years ago

We should introduce a nice convenient way of tracing the performance of FLAME GPU as commits progress.

Both for the examples suite (although changing behaviours would lead to loss of performance potentially) but also for a suite model performance tests with the express purpose of having performance recorded.

Effectively a secondary test suite which records performance for a given commit. Each model should be executed several times, and the average performance recorded (+ the hardware executed on, compiler version(s) and driver version(s))

This may involve a database of results? Possibly use of an external tool to switch

dentarthur commented 4 years ago

AiiDA is an interesting external tool, also relevant for parameter sweeping, ensembles, checkpoints etc:

Cite as: arXiv:2003.12476 [cs.DC] (or arXiv:2003.12476v1 [cs.DC] for this version) [v1] Tue, 24 Mar 2020 12:06:12 UTC (363 KB)

https://aiida.readthedocs.io/projects/aiida-core/en/latest/

AiiDA: automated interactive infrastructure and database for computational science

https://doi.org/10.1016/j.commatsci.2015.09.013

Works with Slurm et al:

https://slurm.schedmd.com/

HPC interface: Move your calculations to a different computer by changing one line of code. AiiDA is compatible with schedulers like SLURM, PBS Pro, torque, SGE or LSF out of the box.

Plugin interface: Extend AiiDA with plugins for new simulation codes (input generation & parsing), data types, schedulers, transport modes and more.

When replacing current XML initialization and output, plugin for AiiDA should be worth considering.

ptheywood commented 3 years ago

pytest-benchmark might be relevant if we want to add a similar thing to the python interface

https://pypi.org/project/pytest-benchmark/