Open espenfl opened 5 years ago
You can use EasyBuild directly from Python in theory, but I'm not aware of people doing this, and I'm not sure how well this would work in practice (I haven't tried this myself).
One important thing you'll definitely need to do is set up the configuration before doing anything:
from easybuild.tools.options import set_up_configuration
set_up_configuration(silent=True)
(see also https://github.com/easybuilders/easybuild-framework/pull/2638)
From that point forward, you can import stuff from the easybuild
namespace and play around.
Generating easyconfigs on the fly as dicitionaries and feeding them in is an interesting idea, I haven't tried that myself, and I'm not sure how easy that would be currently. Maybe just give it a shot, and see what you end up with?
One way could be to:
args
to the main
function in easybuild/main.py
(then you don't even need to run set_up_configuration
first, since main
will take care of that for you)
Or maybe what you're looking for is hooks, see https://easybuild.readthedocs.io/en/latest/Hooks.html.
The start
hook is called early enough to inject easyconfigs from there, but the way it is implemented now it doesn't give you the option to fiddle with the options or arguments that get processed, so some minor changes would have to be made to the start
hook if you'd like to follow that approach...
Thanks for the very quick reply. Greatly appreciated. Okey, I will try this. This question also goes in hand with #2865.
Okey, nearly there following 1. and 2. as an initial approach. I can now pass a list to main
containing the generated easyconfig files. However, how do I pass along the rest of the arguments? Seems args only contains the easyconfig files and not the other arguments. I typically also need include-easyblocks
, detect-loaded-modules
, info
, and robot
.
The args
you can pass to the main
function can both include easyconfig file names and options, so this should work as expected:
main(args['/tmp/example.eb', '--include-easyblocks', '/tmp/*.py'])
The args
list will be passed to the option parser, which will return a list of parsed options and remaining arguments (which typically are easyconfig file names).
On the lookout for strategies to iterate over many combinations of some base software perturbations using Easybuild, an external wrapper seems the way to go. In that context, is there a way to call Easybuild directly from Python? Say that we generate the easyconfig parameters as a dictionary and then execute one Easybuild step with a call to something similar to what is done in
main.py
etc., basically runningeb
without usingrun_cmd
etc.Thanks in advance.