How to use an external key (sftp)

microcreators commented 4 years ago

Pre-Submission checklist

[x] There is no existing issue that already asks for this feature.
[x] The feature already exists in rclone (on your PC or in Termux)
[ ] I am prepared to help make this feature

What version of RCX are you using (About -> App version)?

1.11.4

What problem are you trying to solve?

I successfully imported an existing, encrypted config that has an ssh/sftp remote. Access is granted based on the external ssh key. They key is encrypted and the pass is stored in the config. The key was originally in the same folder as the config, and is as such referenced in the conf file. Rcx copied the config to the internal folder, but not the key and fails to find it, so the remote can't be opened.

What should RCX be able to do differently with help this problem?

Copy the key file as well or use the config file as-is, w/o copying it internally.

ps. probably more of a how-to question, than a request. thanks.

x0b commented 4 years ago

Currently, there is no Android-specific code for external key support, i.e. if the path referenced in the config file is not an absolute path to a file on external storage, it will not work (Cf. https://github.com/x0b/rcx/issues/66#issuecomment-633692339).

I'm currently working on the next version of RCX, which will include Rclone 1.54, and then also support the key_pem config parameter (Cf. https://rclone.org/sftp/#ssh-authentication). With that config option, you can embed your key into the rclone config file.

Until then, you could try to edit your rclone config to point to the Android-specific path of your key file and afterwards import it into RCX. For example:

rclone.conf

[sftp]
...
key_file = /sdcard/keyfile

However, this probably won't work on Android > 9 (Android 10, Android 11), because of tightened security rules. On those devices, the following might work: rclone.conf

[sftp]
...
key_file = /sdcard/Android/data/io.github.x0b.rcx/keyfile

microcreators commented 4 years ago

Putting key into the config also works (even in 1.53). Thanks in advance.

bcliang commented 3 years ago

The latest releases have upgraded the rclone library to 1.55.0, so the key_pem approach should work for SFTP. I tried this out with a configuration file that is known working (using the rclone go application in ubuntu/debian) and wasn't successful. Seems like the connection is successful but directory listing always fails.

Has anyone else tried using a PEM key with success in rcx?

relevant portion of the log

2021-04-27 16:34:41 - 2021/04/27 20:34:29 DEBUG : Using config file from "/data/user/0/io.github.x0b.rcx/files/rclone.conf"
2021/04/27 20:34:29 DEBUG : rclone: Version "v1.55.0-rcx" starting with parameters ["/data/app/~~ryz6SorzuNDBlFZTl7tZ3g==/io.github.x0b.rcx-k7K-cUjfY9Z_R7KMMvkzzw==/lib/arm64/librclone.so" "--cache-chunk-path" "/data/user/0/io.github.x0b.rcx/cache" "--cache-db-path" "/data/user/0/io.github.x0b.rcx/cache" "--config" "/data/user/0/io.github.x0b.rcx/files/rclone.conf" "-vvv" "lsjson" "chi1:"]
2021/04/27 20:34:29 DEBUG : Creating backend with remote "sftp_remote:"
2021/04/27 20:34:30 DEBUG : pacer: low level retry 1/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:30 DEBUG : pacer: Rate limited, increasing sleep to 200ms
2021/04/27 20:34:30 DEBUG : pacer: low level retry 2/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:30 DEBUG : pacer: Rate limited, increasing sleep to 400ms
2021/04/27 20:34:30 DEBUG : pacer: low level retry 3/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:30 DEBUG : pacer: Rate limited, increasing sleep to 800ms
2021/04/27 20:34:30 DEBUG : pacer: low level retry 4/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:30 DEBUG : pacer: Rate limited, increasing sleep to 1.6s
2021/04/27 20:34:31 DEBUG : pacer: low level retry 5/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:31 DEBUG : pacer: Rate limited, increasing sleep to 2s
2021/04/27 20:34:33 DEBUG : pacer: low level retry 6/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:35 DEBUG : pacer: low level retry 7/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:37 DEBUG : pacer: low level retry 8/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:39 DEBUG : pacer: low level retry 9/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:41 DEBUG : pacer: low level retry 10/10 (error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain)
2021/04/27 20:34:41 Failed to create file system for "sftp_remote:": NewFs: couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

The remote is set up with

type = sftp
host = (..hidden..)
port = (..hidden..)
key_pem = -----BEGIN RSA PRIVATE KEY-----\n (...hidden...) \n-----END RSA PRIVATE KEY-----\n
md5sum_command = md5sum
sha1sum_command = sha1sum

x0b commented 3 years ago

@bcliang Can you try to export the configuration from RCX into Termux or onto a PC and running it there? If it works there, please open a new issue in the RCX repository. If it does not work in Termux/on your PC, please open a new issue in the the rclone repo.

bcliang commented 3 years ago

@x0b thanks for the response.

I exported the config as you suggested, and tested the config using Termux on Android and locally. It appears to be the same as the imported file aside from adding a new alias with remote=/storage/emulated/0.

The config worked using rclone (ubuntu) but did not when testing with termux -- same error as I posted above for rcx (build v1.55.0-DEV). Should I post the issue in the termux project, or here?

error couldn't connect SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

output on linux:

❯ rclone --config ~/Downloads/rclone.conf lsd **(hidden)**: -vv
<7>DEBUG : Using config file from "/home/**(hidden)**/Downloads/rclone.conf"
<7>DEBUG : rclone: Version "v1.55.0" starting with parameters ["rclone" "--config" "/home/**(hidden)**/Downloads/rclone.conf" "lsd" "**(hidden)**:" "-vv"]
<7>DEBUG : rclone: systemd logging support activated
<7>DEBUG : Creating backend with remote "**(hidden)**:"
<7>DEBUG : sftp://**(hidden)**/: New connection **(hidden)**->**(hidden)** to "SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.2"
<7>DEBUG : **(hidden)**: Using absolute root directory "/home/**(hidden)**"
          -1 2020-12-10 17:32:02        -1 .cache
          -1 2020-12-25 21:33:46        -1 .config
          -1 2020-12-05 12:13:53        -1 .gnupg
          -1 2020-12-05 13:30:17        -1 .local
          -1 2021-04-23 19:18:06        -1 .ssh
          -1 2021-04-23 19:14:53        -1 conf
          -1 2021-01-05 07:59:46        -1 data
<7>DEBUG : 11 go routines active

x0b commented 3 years ago

The config worked using rclone (ubuntu) but did not when testing with termux -- same error as I posted above for rcx (build v1.55.0-DEV). Should I post the issue in the termux project, or here?

That is relieving (the bug is not RCX-specific), but also somewhat concerning (the bug only appears on Android, which points to bugs in go or the toolchain itself 🙄).

Re: where to file as bug: while Termux lists themself as maintainers for the rclone package, it is effectively maintained by me and the main rclone team of course. I've therefore gone ahead and filed this issue as #134 in this repo. As a next step, I'll try to replicate this on my one machine, and we'll probably have to create a tracking issue in rclone/rclone as well.

x0b / rcx