search

NERSC / shifter

Shifter - Linux Containers for HPC
Other
348 stars 65 forks source link

Mount for shifter on NERSC: II #265

Closed jfroula closed 4 years ago

jfroula commented 4 years ago

I have the same issue as ticket "Mount for shifter on NERSC #231". In that ticket, the solution was the following:

"You need to precreate the mount point in the image. So just add a RUN mkdir /your/mount/path. I feel like this check is to strict so I’m going to keep this open and see if we can allow this."

However, this is not possible in my case. For example, I'm trying to run cromwell (workflow engine) on cori. Cromwell will automatically create the mount points dynamically so the user doesn't know what they will be ahead of time. An example command is

shifter --image=jfroula/jaws-blastplus:1.0.16 -V /global/cscratch1/sd/jfroula/JAWS/jaws-blastplus/RunOnMac/cromwell-executions/runblastplus_sub/7dbb416c-c861-403b-8f5d-d83163751a57/call-shard:/cromwell-executions/runblastplus_sub/7dbb416c-c861-403b-8f5d-d83163751a57/call-shard

FAILED to create volume "to": /var/udiMount//cromwell-executions/runblastplus_sub/7dbb416c-c861-403b-8f5d-d83163751a57/call-shard, cannot create mount points in that location
FAILED to setup user-requested mounts.
FAILED to setup image.
age.

In this example, I don't have a /data directory in the container so

# this works
shifter --image=ubuntu:16.04 -V $(pwd):/data

# but this doesn't (adding slash to data)
shifter --image=ubuntu:16.04 -V $(pwd):/data/

FAILED to create volume "to": /var/udiMount//data/, cannot create mount points in that location
FAILED to setup user-requested mounts.
FAILED to setup im

Can the shifter "check" be relaxed? Thanks

jfroula commented 4 years ago

I found a workaround for this issue. In the Cromwell config file, I can assign the variable dockerRoot to where to Cromwell server will be running. This insures that the mount point within the container exists.