Open paulmelis opened 1 month ago
Just noticed something, if I run tests on all defined node types (--system snellius
in our case) which includes both CPU and GPU nodes, then the dependency warning is not shown and all tests are run. But if I only run on the CPU nodes (--system snellius:genoa
, as used above in the second run) then the tests are skipped due to the dependency check failing.
I tried to reproduce your above setup with some mock tests. The dep_gpu_only
is indeed not reached because it seems that the Blender compilation is not valid for the system that you are trying to run in the first place. Can you try setting the valid systems to be the same for both tests and try to see what happens? This will allow to pass the filtering of ReFrame and run the hook.
I can confirm that changing valid_systems
in Blender_CompileShaders
to list the full range of node types indeed makes the non-GPU tests run without issue. It's slightly suboptimal, as it now makes that base test look like it is needed on the CPU nodes when it is not, but okay.
@paulmelis yeah I don't know if there's a way to get a way with that since to it to reach the hook it should not be filtered out completely.
This is generally a limitation with raw dependencies compared to using fixtures. Fixtures "inherit" the valid_systems
and valid_prog_environs
of their parent based on their scope, so you don't run into the trouble or properly hand-crafting the valid_systems
in your dependencies. My suggestion may require a bit more work but I think it will pay off in the future:
valid_systems
and similarly for environments in valid_prog_environs
. For each system partition and environment in your config, define instead a set of features
and extras
(see here) which you can use as constraints in your valid_systems
and valid_prog_environs
.Blender_CompileShaders
a fixture of the Blender_RIOW
using the environment
scope.
I have a test with an optional dependency, see below. I use the
Blender_CompileShaders
test to force a one-time action (when the NVIDIA driver has changed) of compiling NVIDIA shaders before rendering, which can take quite some time, and I don't want that time to pollute the actual render results ofBlender_RIOW
. But I do want to keep track of the precompile time, hence having it as a separate test that gets logged.The funky thing here is that the
Blender_RIOW
test is run on all of our nodes, including non-GPU ones, while theBlender_CompileShaders
dependency only makes sense on GPU nodes. Hence thevalid_systems = ['snellius:gpu_a100', 'snellius:gpu_h100']
in that class.However, this seems to trip up Reframe somewhat. When I run the test on a GPU node all is well and I can see the
dep_gpu_only()
call being made and returningTrue
:But when I run it on a non-GPU node I get warnings related to dependency resolution,
dep_gpu_only()
never gets called, and two tests are (incorrectly) skipped:Now I can understand that
Blender_CompileShaders
gets filtered out due to itsvalid_systems
not including the system I'm running the test on. But why would this cause theself.depends_on()
inBlender_RIOW
to not calldep_gpu_only()
at all? Shouldn't it evaluate that function first, and only when the dependency is needed check if it can be found?Also interesting to see it list the 2nd test case
Blender_HoleInTheRoof
in the output, which is indeed defined, but I don't ask for it with-n
on the command-line.This is with Reframe 4.6.1
Edit: some wording