'#' symbol ignored by when reading real path on 16.08 and 18.03

[khurtado]

Hello,

I'm trying to use the --volume option on Blue Waters, with shifter v16.08 which works fine, except when I have # in the directory. The same thing happens on a test machine with v18.03. None of the parent directories are symbolic links.

I tried escaping '#', ie.: '#` but that didn't work either. Do you have a workaround for it? I'm doing this as part of an automated process and renaming the directory names created is not a trivial thing to do.

Any ideas or suggestions?

For example:

# Blue Waters
> pwd
/mnt/a/u/sciteam/hurtadoa
> mkdir -p test1#1
> shifter --image=docker:centos:latest --volume=/mnt/a/u/sciteam/hurtadoa/test1#1:/reana --  cat /etc/centos-release
shifter_realpath: failed to lstat /var/udiMount/mnt/a/u/sciteam/hurtadoa/test11
FAILED to find real path for volume "from": /mnt/a/u/sciteam/hurtadoa/test11
FAILED to setup user-requested mounts.
FAILED to setup image.
> module avail shifter

------------------------------------------------------------- /opt/cray/gem/modulefiles -------------------------------------------------------------
shifter/16.08.3-1.0502.8871-cray_gem shifter/16.08.3-2(default)

# Test Machine
$ shifter --image=python:2.7 --volume=/home/khurtado/test1#1:/reana-- cat /etc/issue
shifter_realpath: failed to lstat /var/udiMount/home/khurtado/test11
FAILED to find real path for volume "from": /home/khurtado/test11
FAILED to setup user-requested mounts.
FAILED to setup image.
$ rpm -qa | grep shifter
shifter-imagegw-18.03.0-1.nersc.el7.x86_64
shifter-runtime-18.03.0-1.nersc.el7.x86_64
shifter-18.03.0-1.nersc.el7.x86_64

[craigsteffen]

I'm the User Support person who took the Blue Waters ticket about this issue.

I presume that you're stripping characters from filenames for security reasons. I agree and understand that. However, I think that SILENTLY stripping characters from user input and then operating on that input as if it's valid is very bad and itself very dangerous interface design.

If shifter encounters input that is invalid or a security risk, shifter should stop, flag the input as invalid, complain very specifically to the user what the problem is with the user input, and then abort and exit. It should NOT blindly try to sanitize the input (thus making it very definitely invalid) and then continue to try to operate with now-modified inputs.

[khurtado]

I made a PR allowing '#', in case shifter developers consider this is acceptable. My use case is for mounting a specific path with '#' characters on it that are passed by the HTCondor GT2/5 grid universe as part of the GlobalJobId variable (so, Torque is submitting grid universe condor pilots that are joining an existing condor pool), so I can't really change the pattern.