I'd like to be able to host artifacts on my company's Linux compute server and use SCP, SFTP, etc., to download them with SSH authentication. Unfortunately, this doesn't currently work.
To summarize, we build libcurl using libssh2, which does not honor the SSH config file. Furthermore, neither libssh2 nor libcurl have a config file. (Well, curl does have one, but it doesn't set options by hostname, it's basically just a way to put all of the flags you want to use into a file, and so they encourage you to use different files for each host you want to connect to.)
I've been trying to brainstorm ways to improve things. This is what I have so far:
Add an option to use the scp, sftp, etc., executables to download artifacts with matching URIs. This is how I'm currently working around the issue. See: ManualArtifactDownload.jl.
Add an option to use the curl executable instead of libcurl. My curl executable is able to use my SSH keys, though I do need to add -u dmatz: to the command. That's unfortunate because it seems we'd need a way to programmatically set the username on a per-host basis.
Add our own simple configuration to allow us to programmatically set the username and SSH key paths when we call libcurl. I made a PR to allow us to specify the username when calling download as a baby step in this direction: https://github.com/JuliaLang/Downloads.jl/pull/225.
I'd like to be able to host artifacts on my company's Linux compute server and use SCP, SFTP, etc., to download them with SSH authentication. Unfortunately, this doesn't currently work.
This was discussed a while ago on discourse: https://discourse.julialang.org/t/privately-hosting-binarybuilder-products-with-authentication/88409. There's also a Pkg issue that seems to be making a similar request, but for package downloads using libgit2 instead of artifact downloads: https://github.com/JuliaLang/Pkg.jl/issues/911.
To summarize, we build libcurl using libssh2, which does not honor the SSH config file. Furthermore, neither libssh2 nor libcurl have a config file. (Well, curl does have one, but it doesn't set options by hostname, it's basically just a way to put all of the flags you want to use into a file, and so they encourage you to use different files for each host you want to connect to.)
I've been trying to brainstorm ways to improve things. This is what I have so far:
scp,sftp, etc., executables to download artifacts with matching URIs. This is how I'm currently working around the issue. See: ManualArtifactDownload.jl.curlexecutable instead of libcurl. Mycurlexecutable is able to use my SSH keys, though I do need to add-u dmatz:to the command. That's unfortunate because it seems we'd need a way to programmatically set the username on a per-host basis.downloadas a baby step in this direction: https://github.com/JuliaLang/Downloads.jl/pull/225.