Setting MODULEPATH via config.yaml

[jo-basevi]

Add an option to add module directories to MODULEPATH via config.yaml.

Should close #347

[coveralls]

coverage: 41.843% (+0.05%) from 41.798% when pulling 456a6d6363fe9bbb13126b2944c4e7bf800300b9 on jo-basevi:347-setting-module-use into c8e74243dd27c42129901efbcc82916e8113da84 on payu-org:master.

[jo-basevi]

So a simple solution could be to add a list of directories to add to the MODULEPATH, e.g. moduledirs module use by default prepends to the MODULEPATH, and it doesn't add the directory if it is already in MODULEPATH. So there could be ordering differences. For example, if prior to payu run:

$ module use dir1 
$ module use dir2 
$ echo $MODULEPATH 
dir2:dir1

And in config.yaml the order was

moduledirs:
    - dir2 
    - dir1

MODULEPATH with remain dir2:dir1 while if module use wasn't run before payu run, the order would be dir1:dir2

[jo-basevi]

I prefer a list of additional module directories than defining an entire MODULEPATH, however redefining MODULEPATH will have the benefit of a concretised order of directories E.g. in config.yaml

env:
    MODULEPATH: /g/data/ik11/spack/0.20.1/share/modules/linux-rocky8-cascadelake:/g/data/hh5/public/modules:/etc/scl/modulefiles:/opt/Modules/modulefiles:/opt/Modules/v4.3.0/modulefiles:/apps/Modules/modulefiles

[jo-basevi]

It could be nice grouping the user-defined environment modules and modules directories together however that would add a breaking change to modules option.. e.g.

modules:
   use:
       - /g/data/ik11/spack/0.20.1/share/modules/linux-rocky8-cascadelake
   load:
       - parallelio/2.5.10-intel-2021.6.0

[aidanheerdegen]

It could be nice grouping the user-defined environment modules and modules directories together however that would add a breaking change to modules option

Agreed that does sound like nicest option.

It is possible to keep backward compatibility by testing the return value of popping the modules value off the config dictionary and updating it to the new syntax. This is how it was done when collate transitioned from a boolean to a dictionary of options:

https://github.com/payu-org/payu/blob/c8e74243dd27c42129901efbcc82916e8113da84/payu/fsops.py#L80-L82

[jo-basevi]

Thanks for that suggestion @aidanheerdegen! I've added that in

[jo-basevi]

Nothings changed/added, I just squashed the commits so it's a little cleaner