include test version into output

[adilger]

It would be useful to include the source code version (from git describe) compiled into the binary and printed as part of the results summary. That will allow us to determine which version of the benchmark was run, in case there are bugs found or features added to later versions.

[adilger]

This could be done as part of the Makefile rules, something like:

version.h:
    echo "#define GIT_VERSION \"`git describe`-`git diff src | wc -l`\"" | sed -e 's/   *//g' -e 's/-0//' > version.h.new
    cmp -s version.h version.h.new && rm version.h.new || mv version.h.new version.h

or similar (note 3 spaces in the first sed command to filter the output from wc -l), to produce a version.h file that looks something like:

#define GIT_VERSION "io500-sc19-124-g48eb188"

if checking out an intermediate commit, or a more compact string like:

#define GIT_VERSION "io500-sc19"

if it is exactly on a tagged release, or show how many lines of code have modified in the src/ directory like:

#define GIT_VERSION "io500-sc19-124-g48eb188-17"

if it is running with local modifications.

The same might be useful to add to the IOR and mdtest and pfind output to help check what version of those commands are being run.

The dance with version.h.new is to avoid modifying version.h if the git checkout has not been changed since the last build.

[JulianKunkel]

That is generally a good idea. At the moment, the version is manually hardcoded and output in the result.txt file, e.g.: version = ISC20-testing

That way, we basically need to take care of managing it explicitly and align it with the tag of the code. We could use git revisions if that makes more sense to you.

I believe both can easily be forged if people want to fake it. Let's discuss that at the meeting.

On Thu, Apr 23, 2020 at 8:50 PM adilger notifications@github.com wrote:

This could be done as part of the Makefile rules, something like:

version.h: echo "#define GIT_VERSION \"git describe-git diff src | wc -l\"" | sed -e 's/ *//g' -e 's/-0//' > version.h.new cmp -s version.h version.h.new && rm version.h.new || mv version.h.new version.h

or similar (note 3 spaces in the first sed command to filter the output from wc -l), to produce a version.h file that looks something like:

define GIT_VERSION "io500-sc19-124-g48eb188"

if checking out an intermediate commit, or a more compact string like:

define GIT_VERSION "io500-sc19"

if it is exactly on a tagged release, or show how many lines of code have modified in the src/ directory like:

define GIT_VERSION "io500-sc19-124-g48eb188-17"

if it is running with local modifications.

The same might be useful to add to the IOR and mdtest and pfind output to help check what version of those commands are being run.

The dance with version.h.new is to avoid modifying version.h if the git checkout has not been changed since the last build.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/VI4IO/io500-app/issues/4#issuecomment-618626577, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGW5SRKW66HPS6VUH3U6V3ROCLZJANCNFSM4MPL22NA .

-- Dr. Julian Kunkel Lecturer, Department of Computer Science +44 (0) 118 378 8218 http://www.cs.reading.ac.uk/ https://hps.vi4io.org/ PGP Fingerprint: 1468 1A86 A908 D77E B40F 45D6 2B15 73A5 9D39 A28E

[adilger]

I don't think any of these checks are to stop fake results, but just to allow tracking of submissions that may have hit a specific bug, as was seen in the past (e.g. score calculated incorrectly, timing for each phase was measured differently, etc.) and will likely be hit again in the future.

[JulianKunkel]

Okay, I have pushed something based on your suggestion. I set main.o .PHONY to ensure that the version is updated for every recompilation and injected it by using the compiler flag instead of a header file.

On Thu, Apr 23, 2020 at 9:03 PM adilger notifications@github.com wrote:

I don't think any of these checks are to stop fake results, but just to allow tracking of submissions that may have hit a specific bug, as was seen in the past (e.g. score calculated incorrectly, timing for each phase was measured differently, etc.) and will likely be hit again in the future.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

-- Dr. Julian Kunkel Lecturer, Department of Computer Science +44 (0) 118 378 8218 http://www.cs.reading.ac.uk/ https://hps.vi4io.org/ PGP Fingerprint: 1468 1A86 A908 D77E B40F 45D6 2B15 73A5 9D39 A28E

[adilger]

This was fixed by commit 83cf48b99