Closed chetanyakan closed 3 years ago
mpdude
commented
4 years ago Yes, basically that should work. It’s also described at https://github.com/webfactory/ssh-agent/blob/master/README.md#using-multiple-keys.
Can you see in the action output that both keys have been added to the agent?
Could it be that your remote server fails already after one wrong key has been presented?
chetanyakan
commented
4 years ago Yes. I can see that both the keys were installed. Here's a part of the output from my github actions:
Setup SSH Keys and known_hosts 1s
Run webfactory/ssh-agent@v0.3.0
with:
ssh-private-key: ***
***
env:
GOROOT: /opt/hostedtoolcache/go/1.14.2/x64
Adding GitHub.com keys to /home/runner/.ssh/known_hosts
Starting ssh-agent
Adding private key to agent
Identity added: (stdin) ((stdin))
Identity added: (stdin) ((stdin))
Keys added:
4096 SHA256:nIm8JFZytuVAb/RN1LYFg+ABZrlokJjzpFsKz4FvsO0 (stdin) (RSA)
4096 SHA256:OyXgFF71To+5RlZrSesEMCqae4LqtsiESn0+WjgUYQs (stdin) (RSA)The next step, which includes npm install fails:
npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t ssh://git@github.com/**redacted**/**redacted**.git
npm ERR!
npm ERR! Warning: Permanently added the RSA host key for IP address '140.82.112.3' to the list of known hosts.
npm ERR! ERROR: Repository not found.
npm ERR! fatal: Could not read from remote repository.
npm ERR!
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR!
npm ERR! exited with error code: 128
npm ERR! A complete log of this run can be found in:
npm ERR! /home/runner/.npm/_logs/2020-06-15T10_37_47_273Z-debug.log
make: *** [webapp/.npminstall] Error 1
Makefile:187: recipe for target 'webapp/.npminstall' failed
##[error]Process completed with exit code 2.Could you please try to take npm out of the equation, by running two git clone commands directly from your action?
Use GIT_SSH_COMMAND="ssh -v" git clone <REPO_SSH>, so we get a little more details of where ssh fails.
felixapitzsch
commented
4 years ago +1
I have the same problem with github actions: fetching two private repos with two deployment keys fails.
I just tried to replace npm ci with git clone -v ssh://git@github.com/... , but the error stays the same:
4096 SHA256: ... (RSA) 4096 SHA256: ... (RSA) Cloning into 'my-repo'... Warning: Permanently added the RSA host key for IP address '140.82.113.3' to the list of known hosts. ERROR: Repository not found. fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
[error]Process completed with exit code 128.
felixapitzsch
commented
4 years ago ... using webfactory/ssh-agent@v0.4.0
chetanyakan
commented
4 years ago @felixapitzsch I haven't really had a chance to try the suggestions by @mpdude but I was able to get around the problem by creating GitHub Packages for both my private dependencies and then using those in my package.json file.
mpdude
commented
4 years ago Oh wait!
Where did you add which key?
If you have one repo running the action, and you need to fetch two private dependencies, then you need to create one SSH key.
The private part of it is added as the secret for the repo where the action is run. You just need to add this one key with the action.
The public key part is added in both private repos that you depend upon. Set it up as a deployment key in each one.
The deployment key basically says "when someone comes along and has an SSH key whose public part matches, they may read (or write) to this repo".
Does that make sense?
mpdude
commented
4 years ago Also see this improvement for the README: https://github.com/webfactory/ssh-agent/commit/5ef9e0334a9b8bcfe6bab2d1c15d6f2ebd5a5c8c#diff-04c6e90faac2675aa89e2176d2eec7d8
felixapitzsch
commented
4 years ago I have three private repositories on github, let's call them A, B, and C. A holds an app, while B and C hold libs. A uses B and C as dependencies (in package.json) and has a github action for npm build and sftp deploy on each push to master. Repositories B and C have a deployment key (public key).
My first attempt was trying to use the same pubilc key as a deployment key on B and C. However, github refuses to register the same public key for more than one repository and give some "already in use" error. Therefore, I was forced to add two different deployment keys for B and C.
Repository A holds the two matching private keys as secrets. Actually adding more than one of these private keys will reproduce the issue described above. Adding only one of the private keys will make cloning one of B/C work and the other one fail with permissons denied (as it should be).
If github would just allow reusing the same public key as a deployment key for B and C, everything would be so easy and straight forward ...
mpdude
commented
4 years ago My first attempt was trying to use the same pubilc key as a deployment key on B and C. However, github refuses to register the same public key for more than one repository and give some "already in use" error. Therefore, I was forced to add two different deployment keys for B and C.
I wasn't aware of that, good to know!
Could you please try to add both private keys on your repo A and then try to clone B and C from an action using GIT_SSH_COMMAND="ssh -vv" git clone ...url...?
That should give some SSH verbosity to understand what ssh (run by git) is trying to do.
mpdude
commented
4 years ago Possible explanation: When connecting to the repo where the private key is the second one, GitHub might actually recognize/accept the first key, because it is known. Then, however, the git operation fails because the key is not allowed to access this particular repo.
My guess is the SSH handshake happens before GitHub can know about which repo is requested, so they have no chance of rejecting the "wrong" key in the first place.
mpdude
commented
4 years ago Not sure if we will be able to come up with a neat solution in this action, though 🤕
felixapitzsch
commented
4 years ago 2020-07-02T11:00:27.1232799Z ##[endgroup]
2020-07-02T11:00:27.1656496Z Adding GitHub.com keys to /home/runner/.ssh/known_hosts
2020-07-02T11:00:27.1668218Z Starting ssh-agent
2020-07-02T11:00:27.2279376Z Adding private key to agent
2020-07-02T11:00:27.3188507Z Identity added: (stdin) (ci@mycompany.com)
2020-07-02T11:00:27.3294638Z Identity added: (stdin) (ci@mycompany.com)
2020-07-02T11:00:27.3296086Z Keys added:
2020-07-02T11:00:27.3344158Z 4096 SHA256:uExQ59z4kx1VhLOpci+JjGQaokEA6oA123456L6U/Ds ci@mycompany.com (RSA)
2020-07-02T11:00:27.3346682Z 4096 SHA256:8IsBSAThA123456xZsB197v9TmDdTYS5s46YhSrY2xI ci@mycompany.com (RSA)
2020-07-02T11:00:27.3488956Z ##[group]Run ssh-add -l
2020-07-02T11:00:27.3490067Z [36;1mssh-add -l[0m
2020-07-02T11:00:27.3491120Z [36;1mGIT_SSH_COMMAND="ssh -vv" git clone -v git@github.com:mycompany/repo-b.git[0m
2020-07-02T11:00:27.3492496Z [36;1mGIT_SSH_COMMAND="ssh -vv" git clone -v git@github.com:mycompany/repo-c.git[0m
2020-07-02T11:00:27.3493680Z [36;1mnpm ci[0m
2020-07-02T11:00:27.3494604Z [36;1mnpm run start:build[0m
2020-07-02T11:00:27.3538134Z shell: /bin/bash -e {0}
2020-07-02T11:00:27.3539071Z env:
2020-07-02T11:00:27.3539996Z SSH_AUTH_SOCK: /tmp/ssh-01KYofN3MhoM/agent.2600
2020-07-02T11:00:27.3541017Z SSH_AGENT_PID: 2601
2020-07-02T11:00:27.3542435Z ##[endgroup]
2020-07-02T11:00:27.3654376Z 4096 SHA256:uExQ59z4kx1VhLOpci+JjGQaokEA6oA123456L6U/Ds ci@mycompany.com (RSA)
2020-07-02T11:00:27.3655945Z 4096 SHA256:8IsBSAThA123456xZsB197v9TmDdTYS5s46YhSrY2xI ci@mycompany.com (RSA)
2020-07-02T11:00:27.3673950Z Cloning into 'repo-b'...
2020-07-02T11:00:27.3769277Z OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017
2020-07-02T11:00:27.3770672Z debug1: Reading configuration data /etc/ssh/ssh_config
2020-07-02T11:00:27.3795009Z debug1: /etc/ssh/ssh_config line 19: Applying options for *
2020-07-02T11:00:27.3800905Z debug2: resolving "github.com" port 22
2020-07-02T11:00:27.3807414Z debug2: ssh_connect_direct: needpriv 0
2020-07-02T11:00:27.3808684Z debug1: Connecting to github.com [140.82.113.3] port 22.
2020-07-02T11:00:27.3825566Z debug1: Connection established.
2020-07-02T11:00:27.3826675Z debug1: key_load_public: No such file or directory
2020-07-02T11:00:27.3828258Z debug1: identity file /home/runner/.ssh/id_rsa type -1
2020-07-02T11:00:27.3829440Z debug1: key_load_public: No such file or directory
2020-07-02T11:00:27.3830835Z debug1: identity file /home/runner/.ssh/id_rsa-cert type -1
2020-07-02T11:00:27.3832050Z debug1: key_load_public: No such file or directory
2020-07-02T11:00:27.3833400Z debug1: identity file /home/runner/.ssh/id_dsa type -1
2020-07-02T11:00:27.3834579Z debug1: key_load_public: No such file or directory
2020-07-02T11:00:27.3835970Z debug1: identity file /home/runner/.ssh/id_dsa-cert type -1
2020-07-02T11:00:27.3837541Z debug1: key_load_public: No such file or directory
2020-07-02T11:00:27.3839053Z debug1: identity file /home/runner/.ssh/id_ecdsa type -1
2020-07-02T11:00:27.3840245Z debug1: key_load_public: No such file or directory
2020-07-02T11:00:27.3841625Z debug1: identity file /home/runner/.ssh/id_ecdsa-cert type -1
2020-07-02T11:00:27.3842852Z debug1: key_load_public: No such file or directory
2020-07-02T11:00:27.3844236Z debug1: identity file /home/runner/.ssh/id_ed25519 type -1
2020-07-02T11:00:27.3845400Z debug1: key_load_public: No such file or directory
2020-07-02T11:00:27.3846789Z debug1: identity file /home/runner/.ssh/id_ed25519-cert type -1
2020-07-02T11:00:27.3848279Z debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
2020-07-02T11:00:27.3870821Z debug1: Remote protocol version 2.0, remote software version babeld-ffbef2ae
2020-07-02T11:00:27.3872378Z debug1: no match: babeld-ffbef2ae
2020-07-02T11:00:27.3873437Z debug2: fd 3 setting O_NONBLOCK
2020-07-02T11:00:27.3874697Z debug1: Authenticating to github.com:22 as 'git'
2020-07-02T11:00:27.3876122Z debug1: SSH2_MSG_KEXINIT sent
2020-07-02T11:00:27.3877696Z debug1: SSH2_MSG_KEXINIT received
2020-07-02T11:00:27.3878756Z debug2: local client KEXINIT proposal
2020-07-02T11:00:27.3881511Z debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
2020-07-02T11:00:27.3884894Z debug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519
2020-07-02T11:00:27.3887358Z debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
2020-07-02T11:00:27.3889467Z debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
2020-07-02T11:00:27.3891990Z debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
2020-07-02T11:00:27.3895019Z debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
2020-07-02T11:00:27.3896639Z debug2: compression ctos: none,zlib@openssh.com,zlib
2020-07-02T11:00:27.3897802Z debug2: compression stoc: none,zlib@openssh.com,zlib
2020-07-02T11:00:27.3898910Z debug2: languages ctos:
2020-07-02T11:00:27.3899895Z debug2: languages stoc:
2020-07-02T11:00:27.3900884Z debug2: first_kex_follows 0
2020-07-02T11:00:27.3901894Z debug2: reserved 0
2020-07-02T11:00:27.3902915Z debug2: peer server KEXINIT proposal
2020-07-02T11:00:27.3904844Z debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256
2020-07-02T11:00:27.3906622Z debug2: host key algorithms: ssh-dss,rsa-sha2-512,rsa-sha2-256,ssh-rsa
2020-07-02T11:00:27.3908693Z debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc
2020-07-02T11:00:27.3910975Z debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc
2020-07-02T11:00:27.3913158Z debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
2020-07-02T11:00:27.3915300Z debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
2020-07-02T11:00:27.3916798Z debug2: compression ctos: none,zlib,zlib@openssh.com
2020-07-02T11:00:27.3918192Z debug2: compression stoc: none,zlib,zlib@openssh.com
2020-07-02T11:00:27.3919297Z debug2: languages ctos:
2020-07-02T11:00:27.3920295Z debug2: languages stoc:
2020-07-02T11:00:27.3922461Z debug2: first_kex_follows 0
2020-07-02T11:00:27.3923556Z debug2: reserved 0
2020-07-02T11:00:27.3924894Z debug1: kex: algorithm: curve25519-sha256
2020-07-02T11:00:27.3926229Z debug1: kex: host key algorithm: rsa-sha2-512
2020-07-02T11:00:27.3928761Z debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC:
felixapitzsch
commented
4 years ago Possible explanation: When connecting to the repo where the private key is the second one, GitHub might actually recognize/accept the first key, because it is known. Then, however, the
gitoperation fails because the key is not allowed to access this particular repo.My guess is the SSH handshake happens before GitHub can know about which repo is requested, so they have no chance of rejecting the "wrong" key in the first place.
That would also be my guess. And I don't see any good way to fix this. A colleague suggested to try something like
GIT_SSH_COMMAND='ssh -i private_key_file -o IdentitiesOnly=yes' git clone user@host:repo.git
or
ssh-agent $(ssh-add /somewhere/yourkey; git clone git@github.com:user/project.git)
explicitly specifying single individual keyfiles for each repository to clone. However, I want to use npm directly instead of using git clone. This would force me to specify repo A and B in my package.json as dependencies located in the local file system and use npm pre-install hooks to call git clone with the matching keyfiles in order to clone the repositories to the local filesystem and install from there. This looks super hacky to me and will not be very feasible when using the same package.json for local development.
Any ideas for a different approach?
mpdude
commented
4 years ago Don't worry, you won't have to git clone directly. This is only to better understand and sort things out. If we've got that resolved, npm ... should work just fine, it will run git clone under the hood.
FWIW, I've contacted GH support and maybe they have someone who can help us here.
Are you somewhat familiar with how ssh-agent and stuff works so you could try things locally? That would probably be easier than having to debug everything on the worker node.
mpdude
commented
4 years ago I've tried the ssh-agent behavior described in https://superuser.com/questions/273037/using-the-identityfile-directive-in-ssh-config-when-agentforwarding-is-in-use, and in fact it works that way:
You can use ssh-add -L to list all identities loaded into the agent. If you put each line into a file of its own (say, id-1, id-2, ...), you can then run ssh -i id-2 ..., and then the second identity from the agent will be offered first, before the other ones are tried.
Now the problem is that when ssh is run below git, which is run from, let's say, npm, then you don't have a chance to pass a corresponding -i ... switch to it for every single dependency that needs to be fetched.
So, the trick is to make up URLs for the individual repos. For example, using something like git+ssh://git@repo1.github.com:webfactory/ssh-agent.git#... in your package.json file. Then, additional SSH config can be used to tell SSH that for host repo1.github.com, the actual host to connect to would be github.com and which identity file to use.
This leads to another problem, however: You'd not have those made-up URLs in your package.json file, so you'd need to apply this trick anywhere you'd want to install the dependencies.
Clever ideas, anyone?
mpdude
commented
4 years ago We could write a shim for ssh and make git use it through the GIT_SSH_COMMAND env var.
That shim would have to figure out which repo git is about to clone. It could then list all SSH keys with ssh-add -L, see if one of the keys has the repo name in the comment field, and if so, put the public key identifier in a temporary file that is then passed with an additional -i ... setting to ssh.
Has anybody done this before and knows what invocations such a wrapper would have to expect?
(https://github.com/martinemde/git-ssh-wrapper looks a bit like it?)
mpdude
commented
4 years ago @felixapitzsch Please try the following:
+x. You can keep the file in .github, for example, if it helps you to keep your repo clean. Let's call it ssh-pick-key.#!/bin/bash
# The last argument is the command to be executed on the remote end, which is something
# like "git-upload-pack 'webfactory/ssh-agent.git'". We need the repo path only, so we
# loop over this last argument to get the last part of if.
for last in ${!#}; do :; done
# Don't use "exec" to run "ssh" below; then the trap won't work.
key_file=$(mktemp -u)
trap "rm -f $key_file" EXIT
# Try to pick the right key
ssh-add -L | grep $last > $key_file
ssh -i $key_file "$@"In your workflow, set the GIT_SSH_COMMAND env var to the absolute path of this wrapper file. Probably that would be something like /home/runner/.../.github/ssh-pick-key.
Update your deploy keys to contain your-org/repo-{B,C}.git (that is, orgname/reponame.git) as the key comment.
For this, use ssh-keygen -c -C orgname/reponame.git -f ...keyfile.... It requires you to still have the private deploy key somewhere around. You don't need to change the key in the B/C repos, but have to update the secrets in your A repo to use the new "public" part (which now contains the comment).
Can you follow these instructions and report back?
mpdude
commented
4 years ago @chetanyakan you could also try this please
mpdude
commented
4 years ago GitHub support replied. In short:
Users generally achieve this in a few different ways. The best reference I’ve seen for the various methods you can use is in this gist: https://gist.github.com/gubatron/d96594d982c5043be6d4
Also see https://gist.github.com/vhermecz/4e2ae9468f2ff7532bf3f8155ac95c74 which is linked from the comments there.
I only skimmed both briefly, but it seems they are working without ssh-agent.
Andrewangeta
commented
4 years ago I’m having the same issue except I’m using Swift Package Manager. I’ll just edit the git Config etc.
mpdude
commented
4 years ago @Andrewangeta could you please try if the steps described in https://github.com/webfactory/ssh-agent/issues/30#issuecomment-653383544 work for you?
shaunco
commented
4 years ago If the ssh-agent factory could implement scoping described at https://superuser.com/questions/273037/using-the-identityfile-directive-in-ssh-config-when-agentforwarding-is-in-use , then we could easily do the following:
- uses: actions/checkout@v2
- uses: webfactory/ssh-agent@v0.4.0
with:
ssh-private-key: |
${{ secrets.SSH_REPO1 }}
${{ secrets.SSH_REPO2 }}
${{ secrets.SSH_REPO3 }}
- name: Force git to use ssh and create mappings
run: |
echo remove http header created by actions/checkout
git config http.https://github.com/.extraheader ''
echo add insteadof
git config url."git@repo1.github.com:MYORGNAME/repo1".insteadOf "https://github.com/MYORGNAME/repo1"
git config url."git@repo1.github.com:MYORGNAME/repo1".insteadOf "git@github.com:MYORGNAME/repo1"
git config url."git@repo2.github.com:MYORGNAME/repo2".insteadOf "https://github.com/MYORGNAME/repo2"
git config url."git@repo2.github.com:MYORGNAME/repo1".insteadOf "git@github.com:MYORGNAME/repo2"
git config url."git@repo3.github.com:MYORGNAME/repo3".insteadOf "https://github.com/MYORGNAME/repo3"
git config url."git@repo3.github.com:MYORGNAME/repo1".insteadOf "git@github.com:MYORGNAME/repo3"This would allow all of your references to work locally and in a GitHub action, and captures both ssh and https git references.
One other note, for capturing SSH traffic in nested git calls (like npm->git or go->git), you can use:
env:
GIT_TRACE: 1
GIT_CURL_VERBOSE: 1@shaunco Did you try the approach outlined in https://github.com/webfactory/ssh-agent/issues/30#issuecomment-653383544?
Still hoping that this could pick the right key automatically, without requiring fancy git repo aliases or hostname mappings.
shaunco
commented
4 years ago I did, but was unsuccessful for pulling go packages from private repos (go get -v -t -d ./...). It is possible I did something wrong along the way in your steps.
The route I went down (and PR I provided) seem like a bit of chaos, but it abstracts all the chaos away from anyone using the action. This means they don't have to concern themselves with creating ssh keys in any special way, knowing the path to any extra scripts, or setting any extra env vars. This seems so much more user friendly:
- uses: webfactory/ssh-agent@v0.5.0
with:
ssh-private-key: |
${{ secrets.FIRST_KEY }}
${{ secrets.NEXT_KEY }}
${{ secrets.ANOTHER_KEY }}
repo-mappings: |
github.com/OWNERX/REPO1
bitbucket.com/OWNERY/REPO2
github.com/OWNERX/REPO3All you need to know is which key is for which repo. And while it might feel magic, I tried to fully explain how it works in the readme.
This also solves the problem of git servers booting you in you try too many invalid ssh keys, as each one only gets a single key.
shaunco
commented
4 years ago @mpdude - Given that the .sh script you provided didn't work for go packages, and the PR I submitted (#38) is backwards compatible (all new functionality requires additional options to be used), are there outstanding reasons why you wouldn't want to merge and call this issue fixed?
mpdude
commented
4 years ago I made sure that using a wrapper script as described above works, and with only minor tweaks I have written a blog post about it:
https://www.webfactory.de/blog/using-multiple-ssh-deploy-keys-with-github
Also, I have updated the README file to point to that post and the Gist.
@shaunco Regarding Go packages, I have no experience whatsoever with that. But can it be that go by default uses https instead of ssh (see this FAQ entry)?
shaunco
commented
4 years ago On the Go side, it does use https, which is why part of what I added includes redirecting any attempted https request to an ssh request. Without it, all https requests are (Go or otherwise) require a separate personal access token.
I pretty strongly disagree that a separate, platform specific, script is easier to maintain than the platform agnostic code I added to this action. We use this (now modified) action in about 20 repos, which would imply 20 copies of that script, and that we'd be left to find and maintain a new option for any Windows build environments. That said, it is your action, so I guess we'll carry on using my fork.
shaunco
commented
4 years ago I will also add that this part of your blog post is wrong:
The downside of this approach is that these made-up domain names become part of your dependency declaration file
The method that I used is 100% transparent to any apps making use of git. As I documented in the updates to the README.md, git rewrites to the made up domain names happen magically by git. You use https://github.com/myorg/myrepo or git@github.com:myorg/myrepo and git rewrites it to git@myrepo.github.com:/myorg/myrepo. You do not need to, and should not, use that pseudo host in any of your dependency declarations.
When developing locally, you are using a personal SSH token that already has access to everything, so you don't need any of the rewrites or per-host SSH entries.
So, again, my changes require NO changes to dependency declarations either locally or in CI/CD scripts on GitHub Actions.
ryanzidago
commented
3 years ago @mpdude
I also have the same problem as mentioned above: trying to fetch multiple private repositories in GitHubActions.
I double checked and I can indeed clone (with the git clone command) the two private repositories in the CI.
Here's what I did in detail:
ssh-deploy-key-wrapper.sh script in my projectgit@github.com:some_user/b.git and git@github.com:some_user/c.git..yml CI configuration file I have:
I can successfully clone the two packages separately:
- name: Debugging with Git Clone 1
run: git clone git@github.com:some_user/b.git
- name: Debugging with Git Clone 2
run: git clone git@github.com:some_user/c.gitHowever, when I tried to install the private repositories via Elixir's package manager (Mix), I get the following error:
* Getting some_app (git@github.com:some_user/some_appgit)
fatal: cannot run .github/ssh-deploy-key-wrapper.sh: No such file or directory
fatal: cannot run .github/ssh-deploy-key-wrapper.sh: No such file or directory
fatal: unable to fork
** (Mix) Command "git --git-dir=.git fetch --force --quiet --progress" failed
Error: Process completed with exit code 1.Which is weird because I know that the script is there, I ran cat .gitbub/ssh-deploy-key-wrapper.sh just before the command installing the dependencies for my Elixir project and I could definitely see the content of the file.
mpdude
commented
3 years ago Maybe Mix runs from another cwd? Does it make a difference if you specify an absolute path to the wrapper script?
ryanzidago
commented
3 years ago Hey @mpdude thanks for the quick reply!
In the meantime, I went with @shaunco's fork of the library. If it never makes it to master, then I'll revert back and see if Mix does not run from the cwd.
ecarlson94
commented
3 years ago I've created a PR for this. @mpdude, please review: #53
Edit: new PR against master
dw2kim
commented
3 years ago I am also waiting for #53 to be merged.
I had the exact same issue that @felixapitzsch @chetanyakan and @ryanzidago had. Also, I tried the https://github.com/webfactory/ssh-agent/issues/30#issuecomment-653383544 but did not work for me.
Hope this gets merged in soon
ecarlson94
commented
3 years ago For those having trouble with getting https://github.com/webfactory/ssh-agent/issues/30#issuecomment-653383544 to work, you can find how I got mine to work at Walawren/github-actions-test with terraform.
Things to note:
storage-account-network-rules is a private repository that has a similar integration with another private repository (on-prem-egress-ips)../ prefix is because I set my working directory to ./devstorage-account-network-rules and on-prem-egress-ipsgithub-actions-test and used in my workflow
DaHuoKolmostar
commented
3 years ago I have three private repos, and I want to use their deploy keys as the secret of a forth repo. The proposed solution above in https://github.com/webfactory/ssh-agent/issues/30#issuecomment-653383544 works fine if I run
git clone git@github.com:{user}/{project}.gitHowever it won't work if I run
git clone ssh://git@github.com/{user}/{project}.gitCloning a "ssh:"-prefixed url is useful when working with python projects. Pip is going to deprecate "git+git" and one alternative is "git+ssh", as described in https://github.com/pypa/pip/pull/7543 and https://github.com/pypa/pip/issues/7554. And from the pip install failing log, "git+ssh" is running git clone -q 'ssh://****'.
So my question is if there can be a solution to do git clone ssh://**** with multiple deploy keys.
shaunco
commented
3 years ago @DaHuoKolmostar - using ssh:// (or git_ssh://) works great using the https://github.com/shaunco/ssh-agent/tree/git-repo-mapping branch that is pending as PR #38. We use it for python with a git+ssh://git@github.com/myorg/myrepo.git#subdirectory=some/subdirectory entry in our requirements.txt that is fed to pip install -r requirements.txt.
Our action yml has the following before the pip install.:
- uses: shaunco/ssh-agent@git-repo-mapping
with:
ssh-private-key: |
${{ secrets.SSH_MYREPO }}
${{ secrets.SSH_MYREPOB }}
${{ secrets.SSH_MYREPOC }}
repo-mappings: |
github.com/myorg/myrepo
github.com/myorg/myrepob
github.com/myorg/myrepocWe use that branch of my fork for about 60 repos, so it will be there until it is merged back in to this project ... and likely longer than that as a few other people are using it.
mpdude
commented
3 years ago Please give #59 a try.
mpdude
commented
3 years ago You can try the #59 PR by using @mapped-deploy-keys instead of the current @v0.4.1 for this action.
Instead of using the wrapper script I suggested above, it uses extra Git and SSH configuration to map particular URLs to matching deployment keys.
The good news is that when you created your deployment keys with a key comment denoting the target repository, as it was suggested above and as it was part of @ecarlson94's #53, then you should not need to change anything else.
Using Git + SSH extra configuration has the advantage of being more easily portable to Windows as well; also, since it uses mechanisms build into the underlying commands/tools, we avoid situations where it might not (or not easily) be possible to get the wrapper in place.
I'd be glad if you could give that branch a try and leave a :+1: here if it works for you.
I am trying to install 2 or more private GitHub repositories as an NPM dependency to another project.
In my
package.jsonfile, I have dependencies from GitHub in the following format:The following is a part of my github action yml file:
The
SSH_PRIVATE_KEYs are added as deploy keys to this repo.In the next step, I try to run
npm install. Now based on the order of the private keys, only one of these dependencies installs successfully. I have tried to change the order of these ssh keys, which changes the dependency that cannot be installed.Is it possible to use deploy keys to access both the private repositories in this case? Let me know if you need more information.