Remote path containing backslash not matched on Windows server when using scp.SCPClient.get from Linux client

[vtyw]

When trying to get a file such as V:\test.txt from a Windows host, this exception occurs:

Traceback (most recent call last):
  ...
    scp.get("V:\\test.txt")
  File "/home/user/.virtualenvs/opencv-4.2.0/lib/python3.6/site-packages/scp.py", line 238, in get
    self._recv_all()
  File "/home/user/.virtualenvs/opencv-4.2.0/lib/python3.6/site-packages/scp.py", line 388, in _recv_all
    raise SCPException(asunicode(msg[1:]))
scp.SCPException: scp: 'V:/test.txt': No such file or directory

It looks like backslashes get converted to forward slashes which would make it impossible to copy anything from a Windows server to a Linux client. I also note that spaces in filenames passed to scp.SCPClient.get are treated as if scp is requesting multiple separate files, so it seems like there is both shell interpretation going on AND some unwarranted Linux-style translation of the path happening on a Linux client.

[remram44]

I'm not sure I understand the slash issue, does opening "V:/test.txt" not work on Windows, same as "V:\test.txt"?

Escaping spaces was implemented (#9) and is tested for, so if you're seeing problems there it might be another Windows-specific problem?

[vtyw]

I'm not sure I understand the slash issue, does opening "V:/test.txt" not work on Windows, same as "V:\test.txt"?

I've just realized that scp expects you to use "/" as the path separator even when copying from a Windows machine. So get("V:/test.txt") does work, whereas get("V:\test.txt") does not and gives the misleading error that 'V:/test.txt' doesn't exist.

As for spaces, if I'm trying to copy "V:\test 1.txt", I've tried:

• get('V:/sample\ 1.txt') - 'V:/sample: No such file or directory
• get('"V:/sample 1.txt"') - 'V:/sample 1.txt': No such file or directory
• get('"V:/sample\ 1.txt"') - 'V:/sample/ 1.txt': No such file or directory
• get("'V:/sample\ 1.txt'") - ''''V:/sample: No such file or directory

There is clearly something weird going on with the formatting of the output message, though that may be a separate issue to whether escaping works.

[remram44]

The second version is expected to work (get('"V:/sample 1.txt"')). Are you sure the file exist? Is it "sample" or "test"?

Also all of this might depend on your server implementation (if using cygwin, you might need /cygdrive/v/sample 1.txt, etc)

[vtyw]

Pardon my inconsistency with filename above. That aside, I am sure the file exists. For comparison I can copy it without problem using paramiko.SFTPClient:

sftp.get('V:/sample 1.txt', 'sample 1.txt') # Works
scp.get('"V:/sample 1.txt"') # scp.SCPException

I also just discovered that quoting doesn't seem to be acceptable in the first place, with or without spaces in the filename.

scp.get('V:/test.txt') # Works
scp.get('"V:/test.txt"') # SCPException
scp.get("'V:/test.txt'") # SCPException

[remram44]

I fear I know what is happening.

Do you mind trying the following:

scp.sanitize = lambda x: x
scp.get('V:/sample^ 1.txt', 's.txt')

[vtyw]

>>> scp.sanitize
<function _sh_quote at 0x7f86bb527400>
>>> scp.sanitize = lambda x: x
>>> scp.get('V:/sample^ 1.txt', 's.txt')
scp.SCPException: scp: V:/sample: No such file or directory
>>> scp.get('V:/sample 1.txt', 's.txt')
scp.SCPException: scp: V:/sample: No such file or directory
>>> scp.get('"V:/sample 1.txt"') # Worked!

[remram44]

Ok well what is apparent is:

1. The Windows SSH server does shell escaping like a Windows machine. As there is no way to know what kind of server it is talking to, scp.py assumes UNIX-compatible, and therefore the default value of the sanitize constructor argument is meant for UNIX. You will have to provide your own if you know the server is Windows.
2. I do not understand how shell escaping on Windows works. I wrote this a while back, but it does not seem accurate: https://github.com/VIDA-NYU/reprozip/blob/4d841bd8232c72a8881ea763e12faeed9e97e422/reprounzip-qt/reprounzip_qt/reprounzip_interface.py#L42-L58

[remram44]

I could change the _sh_quote function to use double quotes instead of single quotes, which would make more filename patterns work on Windows (maybe? Does Windows have single quotes?), but that would still be accidental as those are different shells with different rules, and some characters would still cause breakage.

[vtyw]

My understanding is that Windows command-line only supports double-quoted strings.

[remram44]

I will replace the current _sh_quote with this version then, which should improve the situation. I guess I have to revisit reprozip's quoting functions...