search

HPCE / hpce-2017-cw6

2 stars 17 forks source link

Safe termination of search #43

Open jeg114 opened 6 years ago

jeg114 commented 6 years ago

I was wondering if it would be possible to know specifically how search will be terminated ( a signal like SIGTERM or SIGKILL?), not only for cleaning up resources but maybe even to flush completed tests correctly. If neither of this is possible then could there be problems with the uncleaned resources, like missing calls to workload_Destroy?

jeg114 commented 6 years ago

Quite related so Ill also keep it here. In case of partial results of the SmallCruss benchmark, is it okay to print the available tests (and if so do they need to be in the same order) or only whole iterations of the benchmark count towards the throughput metric.

given that the spec says

This metric is how many times the SmallCrush benchmark can be executed per second, which is measured as the number of complete test results printed to stdout, divided by the amount of time the program is allowed to run before the user terminates it

I understand that partial benchmark tests count for the metric? but do not know about the order of tests as it would depend on how you consume the output data

m8pple commented 6 years ago

It will be via a signal - I actually run it under timeout , so the signal sent will be TERM, then there is about a second's grace until KILL is sent.

In terms of the output data, the spec is intended to say that partial results do not count towards the count. So by "number of complete test results", it was intended to mean the complete set of tests for a given execution of the battery (so whole iterations of the benchmark). The reasoning here is that if there is credit for partial tests, then there would be an incentive to just pick the fastest test from the battery, and then execute that one as fast as possible.

Because the output is a csv file, and we have a unique key for the RNG instance, it is possible to disambiguate the results. Actually, I don't think this has come up before, but if lines come out in odd orders, then it is still possible for me to tell when a particular set of tests is complete (as long as the lines aren't torn).