A version of the clembench benchmark is determined by a) the set of games that is run and b) the instances for each game. At the moment, this information is distributed over various places:
Which set of games is run is determined by the calls collected for example in pipeline_clembench.sh
The instances on which they are run is determined by whatever is placed in in/instances.json for each game at the time when the game is run.
Assuming that the set of games constitutes the major version number, and the set of instances the minor version number, it would be good if this information could be consolidated.
One way of doing this would be to make it possible for different instances.json to live in in/, and make them addressable via the version number. At the moment, AFAIK this is not exposed via the code.
A version of the clembench benchmark is determined by a) the set of games that is run and b) the instances for each game. At the moment, this information is distributed over various places:
pipeline_clembench.sh
in/instances.json
for each game at the time when the game is run.Assuming that the set of games constitutes the major version number, and the set of instances the minor version number, it would be good if this information could be consolidated.
One way of doing this would be to make it possible for different
instances.json
to live inin/
, and make them addressable via the version number. At the moment, AFAIK this is not exposed via the code.