smarr / ReBench

Execute and document benchmarks reproducibly.
MIT License
85 stars 24 forks source link

Base RunId comparison and hash on all details #271

Closed smarr closed 3 weeks ago

smarr commented 3 weeks ago

This PR is in preparation for the support of https://github.com/smarr/ReBench/issues/257, which will add machine and denoise configuration support. Though, it’s also useful without, because it allows us to distinguish RunIds that differ in their environment variables, for instance. Or more generally, RunIds that differ based on any property that is not strictly part of the command line, which is was previously used to establish equality.

This as a consequence means, we have a much weaker ability to determine equality of RunIds than before, but I think that’s fine and less surprising/buggy. It consequently undos some of #4. On the other hand, it should be everything needed to resolve #119.

The main changes are a proper implementation of the __eq__, __lt__, and __hash__ methods, as well as the serialization of RunIds into the data file, and deserialization when loading a data file.

This change includes some minor refactorings, because it is split from a too huge and unmanageable patch.

Minor refactorings:

TODO:

coveralls commented 3 weeks ago

Coverage Status

coverage: 53.785% (-0.01%) from 53.798% when pulling a64c9b382c7b7cc4a1b90d9e3520d1af583fbc3b on run-id-equality-based-on-all-props into 22ed497f778b8edd999e357b98eaa2e6debca13d on master.