search

martius-lab / cluster_utils

https://cluster-utils.readthedocs.io/stable/
Other
12 stars 0 forks source link

Running as Python module does not work with singularity containers #69

Closed mseitzer closed 7 months ago

mseitzer commented 7 months ago

In the case of running the target script as a Python module (environment_setup.run_as_module=True), the current way we switch to the target directory is not compatible with singularity. It uses cd, which is a shell builtin and thus not recognized by singularity exec (see https://github.com/martius-lab/cluster_utils/blob/41663a3d8f2ccb682e8d5e8e09327a1fa18e43b5/cluster_utils/job.py#L151). The error is 

FATAL:    "cd": executable file not found in $PATH

One way around would be to wrap whatever singularity should execute inside bash -c <command>. But I don't know what side effects this would have.

mseitzer commented 7 months ago

Actually, I think the cd at that point can just be removed. The directory is switched to the project directory before anyways, and in the case of singularity, the --pwd flag does the same.