As brought up in this issue #379 regarding payu being able to find the path to spack built executables. A potential solution, as suggested by @micaeljtoliveira here, was to add the following in config.yaml:
Where SPACK_ACCESS_OM3_ROOT is set by the access-om3 module.
However currently payu runs module load/use during payu run (Experiment.run()). The paths to the executable are set earlier in the experiment initialisation and if exe is not an absolute path, it prepends the path to the laboratory bin path. The executable paths are checked and added to the manifest during setup in the model driver.
The minimum to get the above solution working, would require moving use/load user modules to Experiment.setup before any Model.setup() is run. It will also require any environment variables to be expanded in the exec_path before the path is checked and added to the manifest. To bypass the absolute path check in initialisation, the exe path will need to start with a forward slash.
I think it could be worth restructuring the Model.exec_path initialisation and setup, so executable paths are set later in Model.setup (vs at payu Experiment initialisation). This means there is no need to ensure there's a starting forward slash in exe as the path will be fully expanded to an absolute path.
As spoke about in the #379, it may be worth adding a check that user module versions are specified. At minimum, it could just be emit a warning if it is using default version.
jo-basevi
commented
2 weeks ago
As brought up in this issue #379 regarding payu being able to find the path to spack built executables. A potential solution, as suggested by @micaeljtoliveira here, was to add the following in
config.yaml:Where
SPACK_ACCESS_OM3_ROOTis set by theaccess-om3module.However currently
payurunsmodule load/useduringpayu run(Experiment.run()). The paths to the executable are set earlier in the experiment initialisation and ifexeis not an absolute path, it prepends the path to the laboratory bin path. The executable paths are checked and added to the manifest duringsetupin the model driver.The minimum to get the above solution working, would require moving
use/loaduser modules toExperiment.setupbefore anyModel.setup()is run. It will also require any environment variables to be expanded in theexec_pathbefore the path is checked and added to the manifest. To bypass the absolute path check in initialisation, theexepath will need to start with a forward slash.I think it could be worth restructuring the
Model.exec_pathinitialisation and setup, so executable paths are set later inModel.setup(vs at payu Experiment initialisation). This means there is no need to ensure there's a starting forward slash inexeas the path will be fully expanded to an absolute path.As spoke about in the #379, it may be worth adding a check that user
moduleversions are specified. At minimum, it could just be emit a warning if it is using default version.