Open LecrisUT opened 9 months ago
I have looked a bit and I found env_modules_python.py
. That's really nice, and I see there are already a great deal of setup implemented there. However, the interface could be improved and packaged directly to PyPI. One change to make it more accessible is to output a lmod json [commands]
instead of raw python interface.
Maybe https://lmodule.readthedocs.io/en/latest/index.html is of interest to you (I've never used it, I just know of it's existence).
Thanks for the link. I did a quick look at the implementation and I do not really like it. The interface goes through the shell script so it can still fail in the Github CI interface and it only provides with simple module load
text commands that you would manually have to chain
I am investigating BaseShell.lua
and I think I can make a PR with a simple json interface soon as a precursor to this
I'll discuss this with Robert, but feel free make a PR and thanks for your input
Thanks, the first part is in #673, I'll need some feedback there of how it should be exported most generally. After that I could make the python interface management, to be included in this project or a one of the third-party ones like lmodule
. Preferably it should be here in order to make downstream packaging easier.
Description
There are a few issues that create friction between Python and Lmod:
subprocess.Popen
do not persist between commandsThese factors make it hard to chain python/shell commands with pushed/poped Lmod module environment
Proposal
Provide a simple Python interface that can call Lmod functions directly. A minimum interface could be:
module list
module load
module unload/purge
The introspection commands could be direct calls to the Lmod interface with some basic output conversion. The main part is how the load/unload interface can look like. Here a
Lmod
class would carry anenv
dictionary with all the Lmod native environment operations likeprepend_path
chained accordingly. Then when requested, these can be evaluated through Lua or equivalent to have the plain python dictionary that can be fed throughsubprocess
Considerations
module getenv
andmodule getscripts
(exporting the environment variable and executed scripts respectively for a given loaded environment) can be provided on the Lmod side, this would simplify writing this interfaceswig
supports making bindings for both languages. That could make such interfaces available to many more other languages as well