rclone and git-annex-remote-rclone support

[yarikoptic]

we use it extensively both in dandi for backup to dropbox. I am tuning up git-annex-remote-rclone ATM and would be great to have consistent way for rclone installation as well. Known to me installation methods for rclone

• downloads: from https://downloads.rclone.org/ (with versions)
• conda
• apt

for git-annex-remote-rclone

• apt
• github (straight from https://github.com/DanielDent/git-annex-remote-rclone/ -- it is just a bash script, there are tagged releases)
• (later) conda -- we need to finalize https://github.com/conda-forge/staged-recipes/pull/18364

in both cases also auto similarly to others

[jwodder]

@yarikoptic rclone has an installation script at https://rclone.org/install.sh that automatically installs the latest version from downloads.rclone.org (though it doesn't support Windows). Should this be used instead of or in addition to datalad-installer installing directly from downloads.rclone.org?

Also, both rclone and git-annex-remote-rclone can be installed via Homebrew on macOS.

in both cases also auto similarly to others

I assume you want the auto order for rclone to be conda > apt > brew > script, and for git-annex-remote-rclone it should be conda > apt > brew > GitHub?

[jwodder]

@yarikoptic Also, what should the "GitHub" method for installing git-annex-remote-rclone be called? Calling it "github" would cause confusion with the datalad/git-annex* methods.

[jwodder]

@yarikoptic

• When installing git-annex-remote-rclone on Windows, if --install-dir is not specified on the command line, where should it be installed? Or should we consider it not supported on Windows?
• When installing git-annex-remote-rclone on Linux or macOS, the most reasonable default install location would seem to be /usr/local/bin, but that usually requires sudo. How should the script tell whether sudo is needed for the install location? Should we just download to a temp location, attempt a normal move, and if that fails with PermissionError, retry with sudo mv?

[yarikoptic]

@yarikoptic rclone has an installation script at https://rclone.org/install.sh that automatically installs the latest version from downloads.rclone.org (though it doesn't support Windows). Should this be used instead of or in addition to datalad-installer installing directly from downloads.rclone.org?

ha -- good find! I love the work rclone author does ;) I would hate to duplicate the effort... may be it could be one of the ways to install? (upstream-install-script ;) ) It has a major shortcoming for our use case -- doesn't support installation of a specific version. See those ATM needed to be supported https://github.com/DanielDent/git-annex-remote-rclone/blob/master/.github/workflows/test.yml#L23

Also, both rclone and git-annex-remote-rclone can be installed via Homebrew on macOS.

cool, so they should have brew method

in both cases also auto similarly to others

I assume you want the auto order for rclone to be conda > apt > brew > script, and for git-annex-remote-rclone it should be conda > apt > brew > GitHub?

I don't remember current order, but I don't see it needing to be different from what we do for git-annex or datalad.

@yarikoptic Also, what should the "GitHub" method for installing git-annex-remote-rclone be called? Calling it "github" would cause confusion with the datalad/git-annex* methods.

why confusion? but could be called DanielDent/git-annex-remote-rclone for symmetry I guess?

• When installing git-annex-remote-rclone on Windows, if --install-dir is not specified on the command line, where should it be installed? Or should we consider it not supported on Windows?

I didn't dare to try/test it on windows yet. It would be ok to not support there for now IMHO

When installing git-annex-remote-rclone on Linux or macOS, the most reasonable default install location would seem to be /usr/local/bin, but that usually requires sudo. How should the script tell whether sudo is needed for the install location? Should we just download to a temp location, attempt a normal move, and if that fails with PermissionError, retry with sudo mv?

Sounds ad-hoc but I do not have a better idea ATM and it sounds like what a human would do -- so must work ;-)

[jwodder]

@yarikoptic

• If we're going to support installation of rclone via downloads.rclone.org directly in order to get version-specific installs, there's really no benefit to be gained from also supporting the install script, since that would just do the same thing, but with less functionality.
• What name should the downloads.rclone.org installation method have?
• When installing rclone on Windows, where should the rclone.exe file be placed by default? Will we also need to fallback to using runas <whatever the Windows equivalent of 'mv' is> on PermissionErrors?

why confusion?

Because the various datalad/git-annex methods also install from GitHub.

but could be called DanielDent/git-annex-remote-rclone for symmetry I guess?

That seems a bit long to type, and we don't have command completion (and I'm not about to implement command completion).

[jwodder]

@yarikoptic Also, the rclone downloads for Linux on downloads.rclone.org include both DEBs and zipfiles; which one should the script install from? Note that the install.sh script only uses zips. (The "deb-url" method can be used to install rclone from a DEB URL, but that requires the user to enter the URL themselves.)

[yarikoptic]

• ok, no script
• downloads.rclone.org is short enough and explicit
• check where that upstream script installs to? Did you mean about sudo ? Then whatever we do for others
• DanielDent/git-annex-remote-rclone people will copy paste from --help output.
• for now can ignore debs, or add downloads.rclone.org:deb to support

[jwodder]

@yarikoptic install.sh doesn't support Windows, and rclone's installation instructions for Windows don't seem to address where to put the executable.