search

microsoft / Git-Credential-Manager-for-Windows

Secure Git credential storage for Windows with support for Visual Studio Team Services, GitHub, and Bitbucket multi-factor authentication.
Other
2.87k stars 590 forks source link

Credential Manager not working. #823

Closed Eddie-Hartman closed 5 years ago

Eddie-Hartman commented 5 years ago

StackO post: https://stackoverflow.com/questions/54170979/git-for-windows-not-able-to-authenticate

A colleague has installed git for Windows via the Visual Studio installed instead of installing the standalone Git for Windows. I've re-installed Visual Studio Community edition and removed the Git installation from Visual Studio.

When trying to authenticate after running git clone *our repo* a Visual Studio sign in prompt appears. So, it is not using the Windows login. When trying to login with the Visual Studio prompt, their Visual Studio login does not work. Also, it opens an OpenSSH sign in prompt directly after the Visual Studio failed login which we also cannot sign into. My installation of git works fine and we tried uninstalling and re-installing it. I've checked environment variables with set >> output.txt to see if they were any different. They are not. I've checked git's credential helper and it is set to "manager" on both machines.

Why is his machine behaving so much different than mine? Why are the Visual Studio and OpenSSH login prompts popping up instead of just using Windows logins? I've even tried just storing logging credentials as plain text and using that for the git login, but that seems to do nothing. (It exhibits the same behavior.)

Any ideas on this? Ideally I'd like to reset every git setting to default and just have it use Windows login.

Update: I looked into how the personal access tokens are used and entered that in the OpenSSH window so that it will connect. Git still prompts for login maybe once, twice, or not at all. It will sometimes work and sometimes not. There doesn't seem to be much consistency and we're seeing the same behavior using VS or command line. Thoughts?

Update: Some logs that might help:

11:05:46.583927 git.c:418               trace: built-in: git clone https://organization@dev.azure.com/organization/BrM/_git/repository
11:05:46.638599 run-command.c:643       trace: run_command: git remote-https origin https://organization@dev.azure.com/organization/BrM/_git/repository
11:05:46.756084 exec-cmd.c:236          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
11:05:46.761174 git.c:675               trace: exec: git-remote-https origin https://organization@dev.azure.com/organization/BrM/_git/repository
11:05:46.763213 run-command.c:643       trace: run_command: git-remote-https origin https://organization@dev.azure.com/organization/BrM/_git/repository
11:05:46.901557 exec-cmd.c:236          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
11:05:47.154374 run-command.c:643       trace: run_command: 'git credential-manager get'
11:05:47.407941 exec-cmd.c:236          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
11:05:47.411328 git.c:675               trace: exec: git-credential-manager get
11:05:47.411328 run-command.c:643       trace: run_command: git-credential-manager get
11:05:47.638866 ...\Common.cs:744       trace: [Main] git-credential-manager (v1.18.3) 'get'
11:05:47.704879 ...\Git\Where.cs:348    trace: [FindGitInstallations] found 1 Git installation(s).
11:05:47.722595 ...Configuration.cs:222 trace: [LoadGitConfiguration] git All config read, 30 entries.
11:05:47.905605 ...\Common.cs:85        trace: [CreateAuthentication] detecting authority type for 'https://organization@dev.azure.com/'.
11:05:48.192481 ...uthentication.cs:231 trace: [DetectAuthority] detected 'https://dev.azure.com/organization/' as Azure DevOps from GET response.
11:05:48.193519 ...uthentication.cs:299 trace: [DetectAuthority] tenant resource for 'https://dev.azure.com/organization/' is {00000000000000000000000000000000}.
11:05:48.193519 ...uthentication.cs:362 trace: [GetAuthentication] MSA authority detected.
11:05:48.246363 ...\Common.cs:205       trace: [CreateAuthentication] authority for 'https://organization@dev.azure.com/' is Microsoft Live.
11:05:48.246363 ...\Common.cs:765       trace: [QueryCredentials] querying 'MicrosoftAccount' for credentials.
11:05:51.026993 ...\Authority.cs:199    trace: [InteractiveAcquireToken] authority host URL = 'https://login.microsoftonline.com/live.com', token acquisition failed.
11:05:51.026993 ...Authentication.cs:94 trace: [InteractiveLogon] failed to acquire token for 'https://organization@dev.azure.com/'.
11:05:51.026993 ...\Common.cs:854       trace: [QueryCredentials] credentials for 'https://organization@dev.azure.com/' not found.
11:05:51.027993 ...\Common.cs:709       trace: [LogEvent] Failed to retrieve Microsoft Live credentials for 'https://organization@dev.azure.com/'.
11:05:51.029960 ...\Program.cs:414      trace: [Get] Logon failed, use ctrl+c to cancel basic credential prompt.
11:05:51.070773 run-command.c:643       trace: run_command: 'git credential-manager get'
11:05:51.231983 exec-cmd.c:236          trace: resolved executable dir: C:/Program Files/Git/mingw64/libexec/git-core
11:05:51.234983 git.c:675               trace: exec: git-credential-manager get
11:05:51.234983 run-command.c:643       trace: run_command: git-credential-manager get
11:05:51.414609 ...\Common.cs:744       trace: [Main] git-credential-manager (v1.18.3) 'get'
11:05:51.491808 ...\Git\Where.cs:348    trace: [FindGitInstallations] found 1 Git installation(s).
11:05:51.498401 ...Configuration.cs:222 trace: [LoadGitConfiguration] git All config read, 30 entries.
11:05:51.654316 ...\Common.cs:85        trace: [CreateAuthentication] detecting authority type for 'https://organization@dev.azure.com/'.
11:05:51.942677 ...uthentication.cs:231 trace: [DetectAuthority] detected 'https://dev.azure.com/organization/' as Azure DevOps from GET response.
11:05:51.942677 ...uthentication.cs:299 trace: [DetectAuthority] tenant resource for 'https://dev.azure.com/organization/' is {00000000000000000000000000000000}.
11:05:51.944717 ...uthentication.cs:362 trace: [GetAuthentication] MSA authority detected.
11:05:51.988268 ...\Common.cs:205       trace: [CreateAuthentication] authority for 'https://organization@dev.azure.com/' is Microsoft Live.
11:05:51.988268 ...\Common.cs:765       trace: [QueryCredentials] querying 'MicrosoftAccount' for credentials.
11:05:54.121044 ...\Authority.cs:199    trace: [InteractiveAcquireToken] authority host URL = 'https://login.microsoftonline.com/live.com', token acquisition failed.
11:05:54.121044 ...Authentication.cs:94 trace: [InteractiveLogon] failed to acquire token for 'https://organization@dev.azure.com/'.
11:05:54.121044 ...\Common.cs:854       trace: [QueryCredentials] credentials for 'https://organization@dev.azure.com/' not found.
11:05:54.121044 ...\Common.cs:709       trace: [LogEvent] Failed to retrieve Microsoft Live credentials for 'https://organization@dev.azure.com/'.
11:05:54.124027 ...\Program.cs:414      trace: [Get] Logon failed, use ctrl+c to cancel basic credential prompt.
11:05:54.168824 run-command.c:643       trace: run_command: bash -c 'test "a$SHELL" '\!'= "a${SHELL%.exe}" || exit 127; cat >/dev/tty && read -r -s line </dev/tty && echo "$line" && echo >/dev/tty'
Eddie-Hartman commented 5 years ago

This is still happening and still needs fixed. I can no longer perform operations via the command line for our organization's repositories.

jeschu1 commented 5 years ago

@Eddie-Hartman can you please contact support for DevOps? They'd be best suited to guide you through some scenarios that will work. It's trying to connect via MSA (first prompt), which fails and then falls back to basic auth (second prompt). When MSA fails that is the expected workflow. One thing to double check is that you select the correct user in the first popup that appears. Also make sure you can access the url you are trying to clone directly in a browser.

Another option to get you immediately unblocked would be to use SSH.

Good Luck!

Eddie-Hartman commented 5 years ago

Thank you for the response. I thought trying SSH to get unblocked would work as a last resort (I did not want to have to make a doc for the whole team to follow to get switched to SSH), but was able to resolve the issue by updating to gcm 1.18.4. It will now use what seems like AAD which is the functionality that we had and wanted before switching out of the previous organization. It was driving me nuts though because of how inconsistent everything was. Sometimes it would authenticate, sometimes it wouldn't. Could not pin down what would make it work or not work even after looking through logs. For now the 1.18.4 exe install seems to be working so fingers crossed it continues to work.

jeschu1 commented 5 years ago

@Eddie-Hartman do you if you are using AAD accounts on a MSA DevOps server. That feature was added in 1.18.4 and would explain exactly what you're seeing.

Also is this a new setup that you got working or an existing setup that stopped working?

Eddie-Hartman commented 5 years ago

We had an existing setup for a company and I am unaware for sure whether or not they were using AAD, but it is very likely. Our team split off to form it's own company no longer part of the larger org and so no longer in their network or Azure DevOps org. I clone mirrored our repos and uploaded them to a new Azure DevOps org that everyone cloned from. Things worked fine for about a week then we started experiencing these inconsistencies until it got to the point that I could no longer authenticate with the repos entirely. I have attempted to turn on AAD for our organization by following this link, but got an error saying that it could not activate AAD in our current state.

jeschu1 commented 5 years ago

@Eddie-Hartman interesting. It sounds like this may indeed be a AAD in MSA scenario. I have had a few validations of this new flow so I'm removing the pre-release tag from 1.18.4.

You may want to followup with support regarding your general auth setup post company split. That said if everything is working you're probably in good shape.

Thanks for the report!

Eddie-Hartman commented 5 years ago

So we've had another team member just start experiencing this issue today. I've updated their credential manager and their command line authenticates fine, but Visual Studio does not even after deleting the tenant.cache file. I feel like this issue should not have been closed. Authenticating for your own repositories should not be this unstable and it's stopping developers from working.

jeschu1 commented 5 years ago

@Eddie-Hartman does everything work from the command line for the user, but fail in Visual Studio? Visual Studio has a packaged version of GCM that we can overwrite once we confirm the command line works.

Eddie-Hartman commented 5 years ago

Yes the command line works. I did not install git for windows via the Visual Studio installer either.

jeschu1 commented 5 years ago

@Eddie-Hartman VS ships with it's own version of git/gcm and the command line uses the installed one on your machine.

The VS team is hoping to have the new version of GCM in a release soon.

We don't like to recommend users alter their VS install as this can become problamatic. However if the user is blocked, they can consider copying the contents of the latest gcm into their VS git folder which should be something like:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer\Git\mingw32\libexec\git-core

Eddie-Hartman commented 5 years ago

So some additional details: My working version in VS GCM is 1.17.1.0 Hers was: 1.14 We both are on latest Visual Studio community edition. After updating hers to 1.18.4.0 she had an error saying that it could not spawn git-askpass.exe. We are currently copying the askpass.exe from our git installation to visual studio. One thing to keep in mind is that the git install is 64 bit (mingw64) and the VS extension is 32 (mingw32). I don't know if that matters.

She still gets the cannot spawn git-askpass.exe: No such file or directory error even though it clearly does exist. This is after clearing tenant.cache again.

jeschu1 commented 5 years ago

@Eddie-Hartman what version of VS was she on? I'd recommend seeing if she can upgrade to latest or the same version as you.

git-askpass should be included as part of the gcm zip.

Eddie-Hartman commented 5 years ago

We are both on the latest version of Visual Studio 15.9.5. I had her update after she started experiencing these issues in case it would resolve the problems. I don't know why our VS GCM would be on different versions.

jeschu1 commented 5 years ago

Interesting. It should be 1.17.1.0 AFAIK. 1.14 is a very old version that doesn't support dev.azure.com. I wouldn't expect this on a 15.9.5 install.

One thing I would try would be to cd into the git-core directory and attempt to use that version of git for a test clone.

Another option to reinstall VS. Since it had the wrong version of GCM I worry it could have other issues with the install.

Eddie-Hartman commented 5 years ago

@jeschu1 Should we install the git for windows individual component when installing VisualStudio? I thought that not doing this would just use the installed git version.

jeschu1 commented 5 years ago

Yes, VisualStudio installs with it's own version of git (which includes gcm).

My suggestion above was to test out the version of git VisualStudio is using by cd'ing into that directory.
That said if the user didn't have 1.17.1.0 in that directory it's very strange and could point to an installation issue.

Eddie-Hartman commented 5 years ago

As you can see, my installation does not have git for windows, but it still has those git core tools. Does this plugin just install the regular git-for-windows installer or does it do something else? I don't understand why Visual Studio has it's own git instead of using the git-for-windows installation I have. image

jeschu1 commented 5 years ago

@Eddie-Hartman VisualStudio needs to understand the output of git and that changes frequently with versions. That's why it needs it's own packaged version.

The VS folks would be best to answer your question on the "Git For Windows" box there. I believe VS ships with a minimal git install and that would include the full distribution.

Eddie-Hartman commented 5 years ago

I zipped up my git core folder from my installation and gave it to her. She was able to do a single git fetch in visual studio successfully and then we tried cloning a repo and was unable to do so. She will uninstall and re-install tonight. We wanted to avoid that because of the amount of downtime we'll have.

jeschu1 commented 5 years ago

@Eddie-Hartman that unfortunately sounds like the best route.

Out of curiosity do you recall what the clone error was?

Just as an FYI your report is the first I've heard of someone having the wrong GCM on VS. We'll definitely keep an ear out for that.

Eddie-Hartman commented 5 years ago

The clone error was the ask-pass error again I believe.

Back to the git install version, it was strange to me to see that as well. This was a fresh install of Visual Studio community edition because it was not even on this machine in December. This machine DID have Visual Studio Professional Edition, but we switched to community in January. I could even see that the date modified for her git-core folder was from January, so I'm not sure how she got an older git-core version installed.

Eddie-Hartman commented 5 years ago

The big issue here is that for the members of the team that this has stopped working for, it just stops completely or for short windows of time it will intermittently work. This has not instilled any confidence in them using git going forward because it just stopped working without any notice. Her install was working even with an older version and we have no idea what caused it to stop working completely. People were downloading zips of folders from AzureDevops and developing out of those and copy-pasting code up as a temporary work-around. I could not even get the credential manager to change what method of authentication it was using, because it would always ask for the a microsoft login (that wouldn't work) twice and then the OpenSSH key from the personal access token (which another temporary workaround was to just let the first two logins fail and keep a PAT key saved on the desktop and copy-paste it in when it asks for it.) I was changing the credential.helper globally, but it seemed to have no effect. I noticed on one developer's machine, it was creating a new PAT for azuredevops each time they tried to log in.

When these things work, they work beautifully, but it just hasn't been working.

jeschu1 commented 5 years ago

Thanks @Eddie-Hartman please keep me updated. It shouldn't work, stop working, and then work again. I know you mentioned the company split. Does your company use a proxy? We've also seen proxies sometimes come into play with auth issues.

If you have another user have the issue where everything was working and then stopped, you can email gcmsupport@microsoft.com (I'm on that alias) and we can collect more data to see whats going on.

It sounds like you know about the PAT workaround where you can cancel out of the AAD prompt and enter the PAT when you get the basic OpenSSH prompt. There is a way to turn that on permanently in GCM but that isn't a great workflow. If a user gets stuck that should allow them to get going.

Eddie-Hartman commented 5 years ago

We do not use a proxy to the best of my knowledge. These are just fairly default setup AzureDevops repos and I was experiencing similar issues whether on the work network, my home network, or a wireless access point. (We thought maybe some firewall was messing things up possibly.) I can reach out the next time this happens via that email and try to get to the bottom of this. I'm all about contributing back, but on company time the focus was just getting back up and running. I'll see if I can take the next machine that experiences these issues home to investigate further without interruptions.

Eddie-Hartman commented 5 years ago

Seems like re-installing Visual Studio has resolved her issues for now. If another case comes up we can do a deep-dive into it to try and figure out the root cause if you like. Thanks for your help. Just wanted to mention that I did post this in the git for windows repo, on stackoverflow, and on reddit and you were the only response. @jeschu1 Thank you for being so helpful. :+1: