Failure to write to /.syncrclone when using restrictive SFTP remote

[jackmawer]

Hi there,

I bumped into a curious issue while trying to sync with a particularly restrictive SFTP remote. Although the home directory is readable as normal, the root directory / returns a permissions error when attempted to be read. It seems that syncrclone causes an issue here, because it doesn't actually just write to .syncrclone in the path specified, but tries to write to the root of the remote from what I understand?

The specific issue with this case of using SFTP is of course that this root directory is unreadable and unwriteable so returns confusing SFTP errors:

2022-12-22 16:44:49: rclone: INFO  : log: Copied (new) to: 4Ogw1_2022-12-22T164028.log
2022-12-22 16:44:49: rclone: INFO  :    13.360 KiB / 13.360 KiB, 100%, 0 B/s, ETA -
2022-12-22 16:44:49: rclone: ERROR : log: Failed to copy: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: rclone: ERROR : Attempt 1/3 failed with 1 errors and: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: rclone: ERROR : log: Failed to copy: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: rclone: ERROR : Attempt 2/3 failed with 1 errors and: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: rclone: ERROR : log: Failed to copy: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: rclone: ERROR : Attempt 3/3 failed with 1 errors and: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: rclone: INFO  :           0 B / 0 B, -, 0 B/s, ETA -
2022-12-22 16:44:49: rclone: Failed to copyto: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: RCLONE ERROR
2022-12-22 16:44:49: CMD ['rclone', 'copyto', '-v', '--stats-one-line', '--log-format', '', '--fast-list', '--no-check-dest', '--ignore-times', '--no-traverse', '/tmp/tmpw_6zo31y/log', 'bx:/.syncrclone/logs/4Ogw1_2022-12-22T164028.log']
2022-12-22 16:44:49: STDOUT and STDERR ERROR : log: Failed to copy: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: ERROR : Attempt 1/3 failed with 1 errors and: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: ERROR : log: Failed to copy: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: ERROR : Attempt 2/3 failed with 1 errors and: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: ERROR : log: Failed to copy: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: ERROR : Attempt 3/3 failed with 1 errors and: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
2022-12-22 16:44:49: INFO  :           0 B / 0 B, -, 0 B/s, ETA -
2022-12-22 16:44:49: Failed to copyto: Put mkParentDir failed: mkdir "/.syncrclone" failed: sftp: "Failure" (SSH_FX_FAILURE)
ERROR. Dumping logs (with debug) to '/tmp/tmpw_6zo31y/log.txt'

The workaround in my case was to make an alias remote pointing directly to the SFTP home folder. However, I wonder if you'd consider writing the .syncrclone directory to the path we are syncing rather than the remote root, or otherwise perhaps explain why it is set up this way?

Thanks :)

[Jwink3101]

However, I wonder if you'd consider writing the .syncrclone directory to the path we are syncing rather than the remote root

This is what is supposed to happen. It shouldn't be writing to the root unless you are using workdirB (or workdirA). if those are set to None (default), then it should be remoteB:.syncrclone.

You truncated a lot of the log but I think this at the very end when it is trying to save the log, right?

---

Can you tell me more about your remote? Can you share the config for bx: (with sensitive info redacted)? Is it a Synology by any chance?

I have a thought as to what may be going on. Let me investigate and I'll report back.

[Jwink3101]

Hi,

I think I found the issue. Can you try to use fix26

python -m pip install git+https://github.com/Jwink3101/syncrclone@fix26

I am not 100% sure if this'll fix it and I can't easily reproduce it on my end but I have an educated guess.

There was a leading / in setting the workdir path. I think this was completely innocuous most of the time since a/b//c is the same as a/b/c and I suspect it got fixed internally.

And on most remotes, X:/path is the same as X:path but, looking at the docs:

Note that some SFTP servers will need the leading / - Synology is a good example of this. rsync.net and Hetzner, on the other hand, requires users to OMIT the leading /.

(BTW, I guessed Synology but I had it flipped. Now I am guessing rsync.net or Hetzner)

This bug may also explain why you are seeing .syncrclone getting written to the wrong place.

WIth that fix, all of my tests but one pass but I think that would fail anyway due to a change in rclone (or something else. I'll look into it later but I wanted to see if this works for you now.)

[Jwink3101]

I merged in the fix and am closing this issue.

First of all, I think that fix was needed either way. It was incorrect being there.

Second, while I haven't heard back, I am 99% sure this will fix it. What you describe, both in terms of the issue and the workaround, is consistent with my fix.

Of course, if this doesn't fix it, please reopen the issue