aces / cbrain

CBRAIN is a flexible Ruby on Rails framework for accessing and processing of large data on high-performance computing infrastructures.
GNU General Public License v3.0
71 stars 42 forks source link

Add task workdir shortening in Singularity containers #1308

Closed prioux closed 1 year ago

prioux commented 1 year ago

This PR allows an admin to turn on a new feature. In all ToolConfigs, there is a new flag Use short workdirs inside Singularity. Default is off.

When turned on, a task run in a Singularity container will see its workdir remapped to a much shorter path such as "/T12345/" (where 12345 is the task's ID).

The system should normally make sure all data files are properly accessible from that new location. This was needed because some stupid C tools crash when a path provided in argument is too long (e.g. some of the internal programs invoked by FMRIPrep).

As an added feature, an admin can go to the work directory of any Singularity-based task and start a Singularity shell simply by invoking:

bash .science.TOOLNAME.1234-1.sh shell

Basically, the scripts named .science.TOOLNAME* can take the single argument shell to bypass the execution of the tool and boot an interactive shell instead.