Failing to install node modules from private repos [using urls]

[jsg2021]

I'm trying to setup a workflow that simply runs npm install/test. I'm having problems with projects that have private github urls. I tried settng token with a PAT that has full repo scopes... no luck. Any insight? npm issues git clone commands... by the description w/ persisting the token, I'd expect this to work😕

{
  "name": "foo",
  "dependencies": {
    "some-private-module": "SomeAcmeCo/some-private-repo"
  }
}

[jsg2021]

I think I found the issue. The docs tell me to use a PAT, but git's extraheader configs want a base64'd "username:pat". I copied the command in the action's error trying to clone the https private repo, and it failed the same way.

I ran the command that this action uses to setup the persisted auth (git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***) and still failed. When I updated the value for token to be the base64 name/token combo it worked... but the action broke. 😕

[ericsciple]

The token is persisted in the local git config so authenticated commands (fetch/push) will work.

If you are running git clone, then git creates a separate .git folder on disk (now a different local git config).

[jsg2021]

Yeah, I switched local to global. Still having issues.

[ericsciple]

The token is added to the local git config using this code

[jsg2021]

i’m wondering if x-access-token as the username portion is somehow not working with clone/ls-remote commands that npm is issuing. 😕 still investigating...

[jsg2021]

It needs to be global. If I add a step before npm install that does:

auth_header="$(git config --local --get http.https://github.com/.extraheader)"
git config --global http.https\:\/\/github.com\/.extraheader "$auth_header"
git config --local --unset-all http.https\:\/\/github.com\/.extraheader

My action passes.

[jsg2021]

assuming actions have an exclusive lock on a container (no other actions can run on it) while running... Can this action grow a global flag?

[jsg2021]

I've made a proposal PR. Thoughts?

[ericsciple]

@jsg2021 a submodule input has been added now. I merged to master, try it out and let me know. Collecting feedback and will update the v2 tag next week.

[jsg2021]

@ericsciple Sorry, master is not working for my action. My PR does work... but after reading your work on the submodule, is probably not secure?

here is the error from my build:

npm ERR! code 128
npm ERR! Command failed: git clone --mirror -q https://github.com/<private>/<org-repo>.git /home/runner/.npm/_cacache/tmp/git-clone-5ad02c85/.git
npm ERR! warning: templates not found in /tmp/pacote-git-template-tmp/git-clone-2b10460c
npm ERR! remote: Invalid username or password.
npm ERR! fatal: Authentication failed for 'https://github.com/<private>/<org-repo>.git/'

[jsg2021]

Here is my action file:

name: Tests

on:
  push:
    branches:
      - master
  pull_request:
    branches:
      - master

env:
  CI: true

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@master
        with:
          token: ${{ secrets.actions_repo_access }}
      - uses: actions/setup-node@v1
        with:
          node-version: "12.x"
      - name: npm config
        run: |
          npm config set loglevel error
          npm config set progress false

      - run: npm install # <-- fails here, git commands to my repos do not have auth
      - run: npm test

[jsg2021]

Your changes would probably work for npm, but you remove the $HOME/global patch after checkout (and only even install it if submodules is true/recursive)... npm needs the auth globally configured to clone private repos during npm install.

[jsg2021]

Maybe add an option to just do the global hack and persist it until the cleanup phase?

[jsg2021]

Any thoughts on this? I really want to start using actions in my org but this is blocking me. I have a fork that works, but since you are trying so hard to prevent tokens from storing in the normal global space, I’m worried my solution is reckless. Are actions not given exclusive control of a container while running?

[ericsciple]

@jsg2021 Global config is avoided due to self-hosted runners. Customers may run many concurrently on a single machine. Whether it's safe depends on whether consumers run multiple runners on a single machine (with same user login).

[jsg2021]

Thanks for the reply. Any thoughts on my requests? (flag to force enable the global hack until cleanup)

[jsg2021]

@ericsciple is the github_workspace the “global scope rewire”? and available even without sub modules? If so, this may solve my blocker! 😊

[jsg2021]

@ericsciple I just tested it. The temporary home is removed before the checkout step completes...before the npm install step... rendering the global git config out of scope for npm. :(

[ericsciple]

I'm hesitant to add a separate input. persist-credentials: global might make more sense.

In the short term i would suggest adding a run step to add the auth token to the global config

[jsg2021]

@ericsciple thanks. I'll have to wait for some solution. If this was just one or two projects I wouldn't mind, but because actions do not have a way to point to a central definition (that I know of) I'd have to add (and then update) 50+ projects.

[quentinchap]

@jsg2021 same issue and I find this https://github.com/marketplace/actions/setup-git-token It can help you.

[jsg2021]

Yeah, I'm trying to avoid setting things globally manually. Since these actions could run on self-hosted machines instead of single-use containers, I want to follow their lead here. What will probably need to happen is the HOME and TEMP/TMP vars will just have to be overwritten per workflow.

[jsg2021]

@ericsciple what are your thoughts on this? I would love to start utilizing GH actions, but this is blocking me from doing so. I can still use my internal build system (so no pressure!), but I look forward to this! 😊

Maybe the solution to all this is a workspace action that privatizes the workflows “global”, “home” & “temp” such that any actions after read/write to a isolated workspace for the workflow. The we can safely run git config set --global?

[ericsciple]

@jsg2021 have you considered SSH instead of PAT? Git supports the env var GIT_SSH_COMMAND. Whereas there isn't a way to set a PAT using an env var (afaik).

If you write the SSH key to $RUNNER_TEMP, the runner will cleanup between jobs.

[jsg2021]

@ericsciple I'd have to upload an ssh identity to the repo secrets? Is there a doc I can read up on?

It would be nice if the actions of a repo inherited the owners/orgs access for which they are in... oh well.

[jsg2021]

I just tested with setting ssh-key, still fails the same way. What do you mean by writing the key to $RUNNER_TEMP? like $RUNNER_TEMP/.ssh/id_rsa?

[jsg2021]

@ericsciple any updates with actions accessing other org private repo? Any new ideas about this?

[jsg2021]

Using webfactory/ssh-agent fixes my issue.