Parser for the spack_spec

[kaanolgu]

Performance Portability post-processing will need the spack_spec parsed, to fish out the compiler name (the thing after %) and the variants (the thing after + is an on-off, which for babelstream is called language, and there are custom ones as well with variant=value).

[ilectra]

It'll be safer to do this with the spack api, and not as a string parsing. But accessing the spack API is a PITA, so @giordano has some ideas of how to do it. Ideally, the parsing of the spack_spec will be done during the benchmarking run, and then its components will be saved in separate columns in the perflog.

[giordano]

I have already a proof of concept of doing something like this in #35. The main ingredients are:

• in benchmarks/modules/utils.py, append to the attribute self.postrun_cmds (note: append, instead of overwriting the attribute as done in that PR, that wasn't a good idea) of the SpackTest class a command which prints out whatever you want
• in the same file, add to the SpackTest class a method decorated with @run_after('run') which captures the quantity you printed out with the post run command above and save into a new attribute of the class
• add the value of the variable captured in this way to the perflog format in benchmarks/reframe_config.py

How to print out the desired information

In #35 I was running the command $(spack -e <path to environment> build-env <spack spec> | grep -oP '^SPACK_CC=\K.*) --version, but that was a very clunky way to do it. A better way is to use Spack Python API (see for example spack tutorial about scripting). Spack isn't usually installed as a regular Python package, so accessing its module isn't straightforward short of fiddling with PYTHONPATH, which can be very fragile, but Spack provides the command spack python to run a Python interpreter which knows how to access the spack module. For example, to show the list of specs in the active environment, assuming it has been already concretized (which is the case for the environment we use in our case) you can do something like

spack -e <path to environment> python -c 'from spack import environment; [x for x in environment.active_environment().added_specs()]'

In practice, you'll need to tweak the command to print exactly what you want.

Caveats

There are a few caveats to be aware of:

• the string you print out should be unique enough that the extract value in postrun command with sn.extractsingle doesn't accidentally captures something else
• there can be in principle multiple specs in an environment. That's not even a theoretical case, we already have a benchmark for which we specify multiple packages as top-level specs: https://github.com/ukri-excalibur/excalibur-tests/blob/a53430b01aaa8fa8326f4cb61db69f540414b983/benchmarks/apps/hpl/hpl.py#L14-L17
• since there can be multiple top-level specs in the environment, you can have also different compilers
• since there can be multiple top-level specs in the environment but the number of columns in the reframe perfolog is fixed, you need to think carefully how to write down all the information you want. An option could be to write the value as a JSON string, which we'll need to read into a dictionary, but maybe someone has better ideas
• need to think what kind of information you want to get: specs names? specs versions numbers? compiler used for each spec? something else?

[ilectra]

Addressed in #262