Closed shahzebsiddiqui closed 1 year ago
this is related to issue https://github.com/buildtesters/buildtest-nersc/issues/157.
@wspear suggested we can auto-generate alias names for every test one idea could be to remove the suite
property under results
such as this test defined in https://buildtest.readthedocs.io/en/devel/buildspecs/spack.html#spack-test
buildspecs:
spack_test_m4:
type: spack
executor: generic.local.bash
description: "Run spack test for m4 package and report results"
tags: [spack]
spack:
root: $HOME/spack
test:
run:
specs: ['m4']
results:
suite: ['spack_test_m4']
option: "-l"
The suite name is name of test spack_test_m4
so if the key spec
is not defined under results we retrieve the suite name otherwise use the specs
.
The question is whether we give user the ability to specify the suite name under spack test run --alias <alias_name>
they must do this under the run
section using options
so it is doable and they can do the following
buildspecs:
spack_test_m4:
type: spack
executor: generic.local.bash
description: "Run spack test for m4 package and report results"
tags: [spack]
spack:
root: $HOME/spack
test:
run:
specs: ['m4']
options: "--alias m4_test"
results:
suite: ['m4_test']
option: "-l"
However i think that it might not be useful having suite
property this only makes sense if user specifies the options
property under run
otherwise they wont know the alias names of hand since spack auto-generates them at runtime. Given that this becomes a problem when re-running the same test multiple times, the command spack test run --alias
will fail if suite name is already present.
Bug Description
the problem is best described in test result https://my.cdash.org/test/67729705?graph=status
The test failure reports
for this buildspec
The issue is that we auto-generate the suite name and since the suite name is taken when rerunning the test this test never ran.
Steps to reproduce the error
Just run
buildtest build
against any spec in this directory https://github.com/buildtesters/buildtest-nersc/tree/devel/buildspecs/e4s/spack_test/perlmutterVersion and HEAD commit
Relevant log output
No response
Post question in Slack
Is there an existing issue