Installing npm dependency from public GitHub repository fails

[TimDaub]

In one of my projects I use simple-caldav which contains the following line in its package.json:

dependencies: {
  "ical.js": "github:TimDaub/ical.js#feat/detect-module-mode-build",
  ...
}

It points to a branch here. I've submitted a PR to the upstream repo, but it seems they're not having much time for maintenance.

Anyways, my GH action in the project that has simple-caldav as a dependency looks like this

# This workflow will do a clean install of node dependencies, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions

name: Node.js CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [10.x, 12.x, 14.x]

    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - run: npm ci
    - run: npm test

However, when it runs npm ci, it fails like this

npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t ssh://git@github.com/TimDaub/ical.js.git
npm ERR! 
npm ERR! Warning: Permanently added the RSA host key for IP address '140.82.113.4' to the list of known hosts.
npm ERR! git@github.com: Permission denied (publickey).
npm ERR! fatal: Could not read from remote repository.
npm ERR! 
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR! 
npm ERR! exited with error code: 128

[Antonio-Laguna]

@TimDaub did you find any workarounds?

[TimDaub]

Nope, I'm quite confused by this problem. E.g. why does it say git ls-remote? Is npm using git internally? I could imagine another notation within package.dependencies could make a difference. But I haven't tested that yet.

[Antonio-Laguna]

Thought the same and even tried that locally and works fine locally

[Antonio-Laguna]

@TimDaub I ended up releasing the forks as my own libraries publicly scoped to NPM since a project was breaking deployments because of this issue. It's a workaround but it's something :)

[carlocorradini]

@TimDaub

Same issue here.

A very annoying problem...

Have you find any solution?

[TimDaub]

ended up releasing the forks as my own libraries publicly scoped to NPM since a project was breaking deployments because of this issue. It's a workaround but it's something :)

Unfortunately, it's not an option I have.

[TimDaub]

What ended up fixing it for me is adding the unknown host in my ssh config before npm ci:

...
- run: mkdir -p $HOME/.ssh/ && echo "140.82.113.4" >> $HOME/.ssh/known_hosts
- run: npm ci
...

It's far from perfect, but works well as a work around for now. Additionally, disabling ssh's key checking via config may be an option too. I prefer to go with this more narrow solution.

Edit: Turns out this won't work all the time as the IPs that the package is requested from change

• 140.82.113.*
• 140.82.112.*
• 140.82.114.*

I've tested adding ranges and ssh-keyscan, but so far I wasn't successful.

Edit2:

I think I finally ended up solving it for good. This is what you'll have to do:

1. Backup your current RSA keypair at ~/.ssh
2. Generate a new RSA keypair on your system ssh-keygen -t rsa -C "your_email@example.com". Ideally don't overwrite your existing keypair at ~/.ssh by entering a custom path.
3. Take the contents of the generated *.pub key and add it to your SSH keys in your GitHub account settings
4. In your repo that has the action, navigate to Settings > Secrets and add SSH_PRIVATE_KEY the contents of the private key file that was generated
5. Then in your repo's workflow file, add the following before -run: npm ci

...
- uses: webfactory/ssh-agent@v0.4.1
   with:
     ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- run: npm ci
...

For more details, check https://github.com/webfactory/ssh-agent

[Siilwyn]

This is still an issue... An alternative solution is replacing the resolved url in the package lock file, for example: git+ssh://git@github.com/zspecza/common-tags.git#946fcbf8cfc1a14c2183ef5a81b23727a2b531e3 becomes: git+https://git@github.com/zspecza/common-tags.git#946fcbf8cfc1a14c2183ef5a81b23727a2b531e3

[natekwilson]

Having the same issue, but within my workflows' docker build step. There has to be easier ways to disseminate SSH agent keys/known hosts info to different contexts when SSH-git actions are so commonplace

[daonb]

This is still a bug. I'm now going to implement the workaround @TimDaub suggested and it'll probably work but still... It hit me just as I released an version and I had no time to fiddle with private keys so github action can read public github URLs. It was a bummer but I released a version whose README starts with "test: failing"

Please fix this as the attached screenshot does not compute.

[akaltar]

For me this issue started occurring when I tried to switch from npm install to npm ci so for some of you switching to npm install may be another workaround.

[Xunnamius]

I fixed this in my workflows by adding an extra step after the actions/checkout@v2 (with persist-credentials: false) step:

      - name: Checkout
        uses: actions/checkout@v2
        with:
          persist-credentials: false

      - name: Reconfigure git to use HTTP authentication
        run: >
          git config --global url."https://github.com/".insteadOf
          ssh://git@github.com/

Changing from SSH to HTTP makes everything work across all workflows using npm ci (which has several benefits over npm install). If you need to authenticate, use a PAT instead of SSH:

git config --global url."https://${{ secrets.GH_TOKEN }}@github.com/".insteadOf ssh://git@github.com/

[TimDaub]

An update on my earlier workaround in this thread. A problem that I've discovered is that according to GitHub settings:

Secrets are not passed to workflows that are triggered by a pull request from a fork.

Hence, it becomes useless when trying to collaborate with others.

[wallind]

I fixed this in my workflows by adding an extra step after the actions/checkout@v2 (with persist-credentials: false) step:

      - name: Checkout
        uses: actions/checkout@v2
        with:
          persist-credentials: false

      - name: Reconfigure git to use HTTP authentication
        run: >
          git config --global url."https://github.com/".insteadOf
          ssh://git@github.com/

Changing from SSH to HTTP makes everything work across all workflows using npm ci (which has several benefits over npm install). If you need to authenticate, use a PAT instead of SSH:

git config --global url."https://${{ secrets.GH_TOKEN }}@github.com/".insteadOf ssh://git@github.com/

this did fix my problem so thank you for that. Out of curiosity though is there any security lost by doing this? I don't care enough to not use this fix for the project I need it on but I am left wondering.

[Xunnamius]

@wallind There is no difference in terms of transport security. HTTPS and SSH rely on similar underlying crypto. Persisting your credentials by adding your secret PAT to the global git config (the last bit of my comment) does have security implications, but it's the default behavior of the Checkout action already (persist-credentials: true) so no security is "lost" per se. If you don't want the PAT hanging around, run some form of post-job cleanup.

[LMBernardo]

Why is this issue closed? I am still having this same issue and the workarounds are just that... workarounds.

[marekdedic]

This is probably caused by npm/cli#2610.

[LMBernardo]

Ah, yeah it is. Thanks.

[Vishv07]

@wallind I have the same issue in the docker file? Any help

[ilanolkies]

Workaround:

sed -i 's/ssh:/https:/g' ./package-lock.json

This will replace all appearances of ssh: for https: in package-lock.json. You can run this before npm install in the Github Actions

[wallind]

fwiw: my team swapped to use Yarn for performance improvements in the CI/CD and now the HTTP protocol workaround isn't needed. Just saying 🤷‍♂️

[GuillaumeFalourd]

I fixed this in my workflows by adding an extra step after the actions/checkout@v2 (with persist-credentials: false) step:

      - name: Checkout
        uses: actions/checkout@v2
        with:
          persist-credentials: false

      - name: Reconfigure git to use HTTP authentication
        run: >
          git config --global url."https://github.com/".insteadOf
          ssh://git@github.com/

Changing from SSH to HTTP makes everything work across all workflows using npm ci (which has several benefits over npm install). If you need to authenticate, use a PAT instead of SSH:

git config --global url."https://${{ secrets.GH_TOKEN }}@github.com/".insteadOf ssh://git@github.com/

Thanks for sharing! 👏🏼

I adapted this solution to a simple action if someone needs something like this in the future in another context.

[dword-design]

I'm having the same issue here https://github.com/dword-design/nuxt-mail/issues/94 and it seems to be an NPM-only thing. Yarn fetches the archived repo snapshots while NPM does some Git stuff with the SSH URL, which is probably the reason for the issue. The SSH to HTTPS replacement workaround is not really an option imho because the end user has to do it with his package-lock.json. So publishing a scoped package is probably the only way to do it.

[ro0gr]

FWIW, using npm@8 has finally fixed this my issue.

[dbougan]

FWIW, using npm@8 has finally fixed this my issue.

Switching from node 14.x to 16.x solved the problem for me (using "npm ci")

[AndrewJHart]

Currently working in dev & prod. Based on @TimDaub original post:

I think I finally ended up solving it for good. This is what you'll have to do:

1. Backup your current RSA keypair at ~/.ssh
2. Generate a new RSA keypair on your system ssh-keygen -t rsa -C "your_email@example.com". Ideally don't overwrite your existing keypair at ~/.ssh by entering a custom path.
3. Take the contents of the generated *.pub key and add it to your SSH keys in your GitHub account settings
4. In your repo that has the action, navigate to Settings > Secrets and add SSH_PRIVATE_KEY the contents of the private key file that was generated
5. Then in your repo's workflow file, add the following before -run: npm ci

For more details, check https://github.com/webfactory/ssh-agent

^ This still works and works well and is the only fix that I feel comfortable with. HTTPS over Github is out of the question imo. So to anyone just stumbling onto this -- I highly recommend you follow his steps.

We use a simple azure PaaS to run a react app, but our component library is not yet published or ready to be published, but does have some awesome functionality for a few components. Ergo, we needed that, along with the deps, then build and deploy. All I had to do was gen the key, add it and configure as mentioned (you can obviously name the secret whatever you want). Here is an edited example:

# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
# More GitHub Actions for Azure: https://github.com/Azure/actions.

name: Build and deploy react/node app to Azure Web App Service

... on & env var removed for brevity ...

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2

      - name: Set up Node.js version & configure GH actions so they can npm install from our own repo
        uses: actions/setup-node@v1
        with:
          node-version: '16.13.2'

      - uses: webfactory/ssh-agent@v0.4.1
        with:
          ssh-private-key: ${{ secrets.SSH_SECRET_KEY }}

      - name: npm install from package-lock, run unit tests, & build
        run: |
          npm ci
          npm run test --if-present
          npm run build:dev

      - name: Upload artifact for deployment job
        uses: actions/upload-artifact@v2
        with:
          name: <app_name>
          path: ${{ env.AZURE_WEBAPP_PACKAGE_PATH }}

  deploy:
    runs-on: ubuntu-latest
    needs: build
    environment:
      name: 'Production'
      url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}

    steps:
      - name: Download artifact from build
        uses: actions/download-artifact@v2
        with:
          name: <app_name>

      - name: 'Deploy App Service'
        id: deploy-to-webapp
        uses: azure/webapps-deploy@v2
        with:
          app-name: <app_name>
          slot-name: 'Production'
          publish-profile: ${{ secrets.PUBLISH_PROFILE }}
          package: .

[andreyvolokitin]

@AndrewJHart the only thing I don't understand is how the github account is relevant here. If CI can be triggered without any action from this specific account — why it even considers this account's SSH key? The SSH connection is between github action runner and github repo, I don't even understand where/when gihub account could become relevant here, but it indeed works (CI is triggered by pushing git tags by my account, I wonder if it will stop working if different account will be pushing tags)

[jdmarshall]

I haven't had any luck getting the workarounds to succeed for this.

There are several very worrying aspects of this bug.

1. npm ci returns a success status when an error has clearly occurred
2. there are no hints in --verbose about how or if it's trying to fetch the files