Closed aekiss closed 1 year ago
There is an undocumented feature** where you can set arbitrary environment variables with payu
So in your case adding
env:
MODULEPATH: '/g/data/ik11/spack/0.20.1/share/modules/linux-rocky8-cascadelake'
should do what you want.
That doesn't mean your suggestion isn't worthwhile, but this is a work-around, and may be sufficient?
** I couldn't find any documentation for it, but would be happy to be corrected. If so it should be added to the documentation.
Thanks @aidanheerdegen that sounds perfect - does this append to MODULEPATH
rather than redefining it?
looks like it redefines MODULEPATH
https://github.com/payu-org/payu/blob/aacfd92570d6f33487ec4de47f9ad8b7a7fa8f12/payu/experiment.py#L461-L476
Yeah it does redefine it. Seems by default:
$ echo $MODULEPATH
/etc/scl/modulefiles:/opt/Modules/modulefiles:/opt/Modules/v4.3.0/modulefiles:/apps/Modules/modulefiles
So yeah, probably have to put the whole lot in there. That is ugly.
Probably worth testing that it works though, and we can decide/work on incorporating something more elegant, like "if string has a leading :
then append rather than replace". That is an existing idiom IIRC.
That is an existing idiom IIRC.
Actually it isn't an existing idiom. But it can be, we just have to make it so!
The implementation is outlined in the docs
https://payu.readthedocs.io/en/latest/config.html#miscellaneous
e.g.
modules:
use:
- /path/to/module/directory
load:
- netcdf-c-4.9.0
- parallel-netcdf-1.12.3
- xerces-c-3.2.3
thanks @aidanheerdegen and @jo-basevi !
Our ACCESS-OM3 configurations (e.g. MOM6-CICE6) rely on the user doing
prior to
payu run
in order to put the right versions of dependencies in theMODULEPATH
environment variable (see https://github.com/COSIMA/access-om3/wiki/Quick-start#running and https://github.com/payu-org/payu/issues/337).This is error-prone, especially as we will likely require particular spack versions to support particular model releases.
Would it be possible to modify payu to support a
moduleuse
list inconfig.yaml
? It already supportsmodules
, so it seems appropriate forconfig.yaml
to also record themodule use
. This way it will show up in the git runlog etc.