Closed alecgunny closed 10 months ago
Hey @alecgunny !
which I believe is the host's python interpreter
I think this is due to the absolute path to the python interpreter in the shebang of the law executable.
With #165 included, can you try adding
[singularity_sandbox]
law_executable = "<the_python_to_use_in_the_container> -m law"
to your law.cfg? I even think that "python -m law" should to the trick. If this works, I would make this the default for all container-based sandboxes.
Hi @riga , working from the latest branch, I've tried explicitly specifying the Python interpreter to be used, but the issue is that this python interpreter doesn't know anything about law
and so can't find it with a simple /usr/local/bin/python -m law
:
/bin/bash: line 1: /usr/local/bin/python -m law: No such file or directory
I can try specifying the full path to the mapped-in law
executable, where my law.cfg
looks like this:
[singularity_sandbox]
allow_binds = true
forward_law = true
law_executable = "/usr/local/bin/python -m /law_forward/py/law/cli/law"
but this doesn't work for some reason either
/bin/bash: line 1: /usr/local/bin/python -m /law_forward/py/law/cli/law: No such file or directory
I guess this probably makes sense anyway: law
outside the container was installed without knowing anything about the dependencies of the libraries insides the container, and vice versa, so there's really no reason to expect these to all be compatible at run time (e.g. when law
attempts to import anything, it will look for the package installed in the container's site-packages
directory first, and there's no guarantee that the correct version will be there).
It seems like the most sensible solution in the short term is to just install law
inside all of our containers, that way we'll know it's present and compatible with the rest of the libraries in the environment.
This would indeed solve things and be a little more robust against incompatibilities between existing and forward python packages, but I'd also be curious why the forwarding doesn't seem to work in the first place.
Just as a test, could you add
[logging]
law.sandbox.base: DEBUG
to your law.cfg
(on the latest master)? This will show you the full sandbox command right after the entering sandbox
banner. I suspect that the PYTHONPATH is not properly set.
Closing this for now, but feel free to open again to continue the discussion if needed :+1:
Full reproducing code can be found here, this issue is concerned with issue 1 on that repo's README. In particular, when trying to run code installed at container build time when using a singularity sandbox, the python interpreter, which I believe is the host's python interpreter, is unable to find the corresponding libraries, even when
PYTHONPATH
is set viasandbox_env
or even viasys.path.insert
inside the container. Taking the linked example (which includesprint
statements indicating the desired and actual values ofPYTHONPATH
), you should see output like:where
greeter
is the name of a library that got installed at build time that evidently can't be found at run time. If you build the corresponding container from the repro code, it's easy to see that this command should work: