npm / npm

This repository is moving to: https://github.com/npm/cli
http://npm.community
17.53k stars 3.02k forks source link

npm fails to install private git repo #11567

Closed gaurav21r closed 7 years ago

gaurav21r commented 8 years ago

Everything was working fine and npm was prompting me for Auth Details, but suddenly when I installed a private repo from github directly via url ie

$ npm install https://github.com/gaurav21r/privaterepo#master

$ npm install https://github.com/gaurav21r/privaterepo#master
npm WARN addRemoteGit Error: Command failed: git -c core.longpaths=true config --get remote.origin.url
npm WARN addRemoteGit
npm WARN addRemoteGit     at ChildProcess.exithandler (child_process.js:213:12)
npm WARN addRemoteGit     at emitTwo (events.js:100:13)
npm WARN addRemoteGit     at ChildProcess.emit (events.js:185:7)
npm WARN addRemoteGit     at maybeClose (internal/child_process.js:827:16)
npm WARN addRemoteGit     at Socket.<anonymous> (internal/child_process.js:319:11)
npm WARN addRemoteGit     at emitOne (events.js:90:13)
npm WARN addRemoteGit     at Socket.emit (events.js:182:7)
npm WARN addRemoteGit     at Pipe._onclose (net.js:471:12)
npm WARN addRemoteGit  git+https://github.com/gaurav21r/privaterepo.git#master resetting remote C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-master-a01cf8c5 because of error: { [Error: Command failed: git -c core.longpaths=true config --get remote.origin.url
npm WARN addRemoteGit ]
npm WARN addRemoteGit   killed: false,
npm WARN addRemoteGit   code: 1,
npm WARN addRemoteGit   signal: null,
npm WARN addRemoteGit   cmd: 'git -c core.longpaths=true config --get remote.origin.url' }
npm ERR! git clone --template=C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\_templates --mirror https://github.com/gaurav21r/privaterepo.git C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-master-a01cf8c5: Cloning into bare repository 'C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-master-a01cf8c5'...
npm ERR! git clone --template=C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\_templates --mirror https://github.com/gaurav21r/privaterepo.git C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-master-a01cf8c5: remote: Invalid username or password.
npm ERR! git clone --template=C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\_templates --mirror https://github.com/gaurav21r/privaterepo.git C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-master-a01cf8c5: fatal: Authentication failed for 'https://github.com/gaurav21r/privaterepo.git/'
npm ERR! Windows_NT 10.0.10240
npm ERR! argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "https://github.com/gaurav21r/privaterepo#master"
npm ERR! node v5.6.0
npm ERR! npm  v3.5.2
npm ERR! code 128

npm ERR! Command failed: git -c core.longpaths=true clone --template=C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\_templates --mirror https://github.com/gaurav21r/privaterepo.git C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-master-a01cf8c5
npm ERR! Cloning into bare repository 'C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-master-a01cf8c5'...
npm ERR! remote: Invalid username or password.
npm ERR! fatal: Authentication failed for 'https://github.com/gaurav21r/privaterepo.git/'
npm ERR!
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>
npm ERR! Please include the following file with any support request:
npm ERR!     D:\xampp\htdocs\aspire-core\npm-debug.log
othiym23 commented 8 years ago

npm doesn't prompt for passwords for repositories hosted via HTTPS. Try switching to the GitHub shortcut syntax (gaurav21r/privaterepo#master in this case), which tries all of ssh, HTTPS, and Git, and will use your ssh credentials, or embedding the username and password in the repository URL (which has security implications).

If for whatever reason neither of those options are available to you, you may be stuck. Plugging in support for specific Git auth strategies that doesn't conflict with npm's need to run as a cross-platform, standalone installer is pretty tricky, and while it does support installing from private repositories, the design is biased in favor of ssh auth, due to the seamless compatibility with tools like ssh agents.

Thanks for your time, and let us know how it goes!

gaurav21r commented 8 years ago

@othiym23 Thansk for your response! I tried the shortcut syntax, still doesn't work :cry: I've seamlessly installed private repos through npm till yesterday!

I recently did npm adduser to publish a package to npm and I think that is the only change so far. I have uninstalled and installed npm & node on Win10 since then and also ran npm whoami. Still no luck. This ->

npm ERR! Command failed: git -c core.longpaths=true clone --template=C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\_templates --mirror https://github.com/gaurav21r/privaterepo.git C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-a01cf8c5
npm ERR! Cloning into bare repository 'C:\Users\Gaurav Ramanan\AppData\Roaming\npm-cache\_git-remotes\git-https-github-com-gaurav21r-privaterepo-git-a01cf8c5'...
npm ERR! remote: Invalid username or password.
npm ERR! fatal: Authentication failed for 'https://github.com/gaurav21r/privaterepo.git/'

Seems to be a dangerous error! is it possible that npm is using npm credentials to login to GitHub?

othiym23 commented 8 years ago

is it possible that npm is using npm credentials to login to GitHub?

It's unlikely, as npm doesn't pass credentials to Git at all – it hands the auth process off to Git completely (unless the username and password have been incorporated into the Git URL). If you're not using an ssh agent on Windows, that may be part of your problem. It's been a little while since I set it up, but I believe I got credential sharing using Pageant to do agent forwarding on Windows. Sorry for not noticing that you were using Windows originally!

gaurav21r commented 8 years ago

@othiym23 I can live with the SSH solution, except, My colleague on the same network with the same version of npm, node AND git installing the same package is able to do it! I think it is some sort of config issue. Sorry about this darn issue but its just driving me cRaZZy since morning!

Btw, this is exactly what's coming for me too #8552

igabesz commented 8 years ago

+1 for this one. We cannot use our private repos over Windows which really sucks.

> npm i bitbucket:owner/repo
...
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
igabesz commented 8 years ago

Solving this issue could also solve #11382, which is closed right now, but not really solved. Also #9332 which was closed in a cleanup, but a few months ago the owner wanted to open it again. (No, it has not been opened.)

tconroy commented 8 years ago

+1, currently facing this issue when trying to npm install from a private repo on a GitHub Enterprise instance. unfortunately the github shorthand syntax won't work for us since it looks at github.com, not our locally hosted enterprise edition.

npm ERR! git fetch -a origin (http://github.<private enterprise repo>.git) remote: Invalid username or password.
npm ERR! git fetch -a origin (http://github.<private enterprise repo>.git) fatal: Authentication failed for 'https://github.<private enterprise repo>.git/'
npm ERR! git fetch -a origin (http://github.<private enterprise repo>.git) remote: Invalid username or password.
npm ERR! git fetch -a origin (http://github.<private enterprise repo>.git) fatal: Authentication failed for 'https://github.<private enterprise repo>.git/'
npm ERR! git fetch -a origin (http://github.<private enterprise repo>.git) remote: Invalid username or password.
npm ERR! git fetch -a origin (http://github.<private enterprise repo>.git) fatal: Authentication failed for 'https://github.<private enterprise repo>.git/'
npm ERR! Darwin 15.4.0
npm ERR! argv "/Users/tconroy/.nvm/versions/node/v5.5.0/bin/node" "/Users/tconroy/.nvm/versions/node/v5.5.0/bin/npm" "install"
npm ERR! node v5.5.0
npm ERR! npm  v2.14.22
npm ERR! code 128

npm ERR! Command failed: git fetch -a origin
npm ERR! remote: Invalid username or password.
npm ERR! fatal: Authentication failed for 'https://github.<private enterprise repo>.git/'
npm ERR!
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /npm-debug.log

In our package.json file, we declare the dependencies like so:

"package-name": "git+http://github.<private enterprise repo>.git#feature/npm",
bs-thomas commented 8 years ago

I wish this could be seriously looked at & fixed.

We are forced (well almost forced) to place the username and password (which need to be shared credentials) directly into the Git repo. Now some staff left, and he seems to be still pulling using the same credentials.

I want to disable his pulling!

igabesz commented 8 years ago

Note that this feature is somewhat against the interest of the NPM team since if one uses their private Github or Bitbucket repo then they won't need a private NPM repo -- ergo the NPM team won't get paid. And some of us do complain here while others just buy their private NPM repo because it works.

I'm getting the feeling that the NPM team is having some second thoughts besides enthusiastically supporting the JS stack development. (And also note that this question is not "open source" related since we're talking about closed proprietary repos.)

gaurav21r commented 8 years ago

@igabesz I don't really think commercial interests are at play here. From whatever I've seen of the NPM team, they are quite transparent. Infact if they have even given a note to host our own registry! https://docs.npmjs.com/misc/registry

lanekatris commented 8 years ago

;TLDR (Windows OS issue) Manually delete %appdata%\Roaming\npm-cache

I had similar errors. One thing I tried was npm cache clean, which took used a lot of CPU and never "finished". So I went to %appdata%\Roaming\npm-cache and tried to delete, I couldn't.

I had to right click the folder >> Properties >> Security >> Advanced >> Permissions tab I clicked my username, then I checked "Replace all child object permission entries...", then clicked Apply

For some reason my git remotes in the cache didn't have permissions under the security tab. image

tconroy commented 8 years ago

@loonison101 interesting. Although permissions appear to be fine on my end ( OSX ) and I'm still encountering the issue.

hayesmaker commented 7 years ago

+1. Another Node + Windows fail. I can clone the private repo locally, and have attempted to npm link that repo into my project, to try to make it skip that install.. But it still tries and fails to secure git permissions (same error as the original post).. Which I must have configured correctly, since I can git clone (via ssh) the private repo.

igabesz commented 7 years ago

We have a "less-than-a-workaroud" with private hosting. Might be good for some, however, it does not solve this issue.

You can -- and maybe should -- separate the repository containing the source code, and the built packages. You can create your own NPM system with Sinopia -- and you can set that only the @my-company/* packages would be hosted there. So, for most of the packages you can use the regular NPM system and for your own packages you can use your own NPM hosting. It does require a hosting and some config but this results in a clean configuration.

eric-burel commented 7 years ago

@othiym23 you said "npm doesn't prompt for passwords for repositories hosted via HTTPS". Why this behaviour ? I did not find documentation about this.

I had a 403 authentication issue (HTTPS aws codecommit repo), that I could not solve since I was never asked my username and password. My workaround is to manually trigger the faulty git command so that the prompt appears, and to cache my credential, but obviously this is quite dirty.

Note that I cannot use SSH or change anything to the package.json, I must manage to install the private package through HTTPS. Those are only workarounds anyway.

silentbugs commented 7 years ago

So what are the current possible workarounds, if any? I'm also using nodejs on Windows, with my ssh keys properly set up (I can git clone the same package just fine). Npm tries to clone the packages improperly and then install them from the npm-cache dir, which obviously fails. Reading this thread, it was my understanding that using ssh keys should work?

Related output:

npm ERR! Windows_NT 10.0.14393
npm ERR! argv "D:\\nodejs\\node.exe" "D:\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "git+ssh://git@bitbucket.org:some/private/repo.git#0.0.1"
npm ERR! node v7.6.0
npm ERR! npm  v4.1.2
npm ERR! code 128

npm ERR! Command failed: git -c core.longpaths=true clone C:\Users\Username\AppData\Roaming\npm-cache\_git-remotes\git-ssh-git-bitbucket-org-some-private-repo-git-0-0-1-c61916de C:\cygwin64\tmp\npm-2272-72d6150e\git-cache-27f0fcf5\c32a4de78b2fc8fb42994726398e73e8e34595a6
npm ERR! Cloning into 'C:\cygwin64\tmp\npm-2272-72d6150e\git-cache-27f0fcf5\c32a4de78b2fc8fb42994726398e73e8e34595a6'...
npm ERR! fatal: '/cygdrive/c/Users/Username/AppData/Roaming/npm-cache/_git-remotes/git-ssh-git-bitbucket-org-some-private-repo-git-0-0-1-c61916de/C:\Users\Username\AppData\Roaming\npm-cache\_git-remotes\git-ssh-git-bitbucket-org-some-private-repo-git-0-0-1-c61916de' does not appear to be a git repository
npm ERR! fatal: Could not read from remote repository.
npm ERR!
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR!
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     C:\cygwin64\home\Username\npm-debug.log
1 Username@Computer ~ %

As mentioned, git cloning the exact same repository works just fine, so does ssh git@bitbucket.org. As you can see I'm also using cygwin.

npm-robot commented 7 years ago

We're closing this support issue as it has gone three days without activity. The npm CLI team itself does not provide support via this issue tracker, but we are happy when users help each other here. In our experience once a support issue goes dormant it's unlikely to get further activity. If you're still having problems, you may be better served by joining package.community and asking your question there.

For more information about our new issue aging policies and why we've instituted them please see our blog post.

adambowles commented 7 years ago

I started getting this error in GitLab's CI when npm v5 was released. Fixed it by appending #master to the package.json entry:

"package-name": "git+ssh://git@git.domain.com/libraries/package-name.git#master",

Oddly though, I had no trouble installing on my machine (Fedora 22, npm 5.0.3) without the #master. It was only the Docker container used by GitLab CI that was failing

thom-nic commented 7 years ago

I'm seeing this with npm@3.8 and npm@4 (nodejs v5.11).

This works:

npm i git+ssh://git@github.com:myOrg/myProj#v2.x

ssh git@github.com also works as expected.

The same line in package.json:

    "myProj": "git+ssh://git@github.com:myOrg/myProj#v2.x",

fails:

$ npm i
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror https://github.com/myorg/myProj.git /home/user/.npm/_git-remotes/git-https-github-com-myorg-myProj-git-123123123: Cloning into bare repository '/home/user/.npm/_git-remotes/git-https-github-com-myorg-myProj-git-123123123'...
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror https://github.com/myorg/myProj.git /home/user/.npm/_git-remotes/git-https-github-com-myorg-myProj-git-123123123: remote: Invalid username or password.
npm ERR! git clone --template=/home/user/.npm/_git-remotes/_templates --mirror https://github.com/myorg/myProj.git /home/user/.npm/_git-remotes/git-https-github-com-myorg-myProj-git-123123123: fatal: Authentication failed for 'https://github.com/myorg/myProj.git/'
Cobertos commented 6 years ago

Windows: If you have git bash installed, try running it in there. It was able to properly prompt me for the password to my password protected RSA key.

I was originally trying to run this command in cmd but this wasn't working. It was strange because I could git clone ... just fine.

kuashe commented 6 years ago

Ran into the same issue today

Make sure Git is installed on the same disk as npm

I had Git installed in D:\Program Files\ and npm in C:\Program Files\ which causes an issue on windows.

I uninstalled Git and reinstalled it in C:\Program Files , it works now.

Size-of commented 6 years ago

i had solved a similar problem, when i used npm install a private git repo, and it fails, error likes: npm ERR! fatal: Authentication failed for 'https://....' i noticed that it need username and password when i used git clone, so i add a command : git config --global credential.helper store and it works.

abhishekDeshmukh74 commented 6 years ago

It's getting the same error through gitbash also :/