search

microsoft / Git-Credential-Manager-for-Windows

Secure Git credential storage for Windows with support for Visual Studio Team Services, GitHub, and Bitbucket multi-factor authentication.
Other
2.87k stars 589 forks source link

git-credential-manager (get/store) hangs on push #871

Closed sebres closed 5 years ago

sebres commented 5 years ago

Which Version of GCM are you using ? 1.18.x .. 1.20.0

Which service are you trying to connect to Every.

Expected behavior it doesn't hang. :)

Actual behavior git-credential-manager.exe hangs immediately, no matter it started from git or directly via command line; config parameter modalprompt was not set, setting of modalprompt to true/false or unset it again changing nothing.

Set the env variables GCM_TRACE=1 and GIT_TRACE=1 and run your git command. Redact any private information and attach the log

Sometimes it hangs with get sometimes with store, like here:

  > git push ...
  23:25:14.228671 exec-cmd.c:237          trace: resolved executable dir: C:/Dev/Git/mingw64/bin
  23:25:14.329671 git.c:439               trace: built-in: git push --progress gh-origin test/travis-cmdMZ-timing
  23:25:14.381671 run-command.c:663       trace: run_command: git remote-https gh-origin https://github.com/path-to-repo/repo.git
  23:25:14.423671 exec-cmd.c:237          trace: resolved executable dir: C:/Dev/Git/mingw64/libexec/git-core
  23:25:14.428671 git.c:702               trace: exec: git-remote-https gh-origin https://github.com/path-to-repo/repo.git
  23:25:14.428671 run-command.c:663       trace: run_command: git-remote-https gh-origin https://github.com/path-to-repo/repo.git
  23:25:14.473671 exec-cmd.c:237          trace: resolved executable dir: C:/Dev/Git/mingw64/libexec/git-core
  23:25:15.053671 run-command.c:663       trace: run_command: 'git credential-manager get'
  23:25:15.226671 exec-cmd.c:237          trace: resolved executable dir: C:/Dev/Git/mingw64/libexec/git-core
  23:25:15.230671 git.c:702               trace: exec: git-credential-manager get
  23:25:15.230671 run-command.c:663       trace: run_command: git-credential-manager get
  23:25:15.371671 ...\Common.cs:744       trace: [Main] git-credential-manager (v1.20.0) 'get'
  23:25:15.441671 ...\Git\Where.cs:348    trace: [FindGitInstallations] found 1 Git installation(s).
  23:25:15.448671 ...Configuration.cs:222 trace: [LoadGitConfiguration] git All config read, 33 entries.
  23:25:15.451671 ...\Common.cs:579       trace: [LoadOperationArguments] http.proxy = 'http://proxy:81'.
  23:25:15.452671 ...tionArguments.cs:605 trace: [SetProxy] successfully set proxy to 'http://proxy:81/'.
  23:25:15.512671 ...\Common.cs:85        trace: [CreateAuthentication] detecting authority type for 'https://github.com/'.
  23:25:15.531671 ...uthentication.cs:151 trace: [GetAuthentication] created GitHub authentication for 'https://github.com/'.
  23:25:15.531671 ...\Common.cs:176       trace: [CreateAuthentication] authority for 'https://github.com/' is GitHub.
  23:25:15.532671 ...\Common.cs:765       trace: [QueryCredentials] querying 'GitHub' for credentials.
  23:25:15.546671 ...uthentication.cs:175 trace: [GetCredentials] credentials for 'https://github.com/' found.
  23:25:16.653671 ...\Authority.cs:200    trace: [ValidateCredentials] credential validation for 'https://github.com/' succeeded.
  23:25:16.655671 ...\Common.cs:873       trace: [QueryCredentials] credentials for 'https://github.com/' found.
  23:25:16.656671 ...\Common.cs:709       trace: [LogEvent] GitHub credentials for 'https://github.com/' successfully retrieved.
  23:25:17.653671 run-command.c:663       trace: run_command: 'git credential-manager store'
  23:25:17.763671 exec-cmd.c:237          trace: resolved executable dir: C:/Dev/Git/mingw64/libexec/git-core
  23:25:17.766671 git.c:702               trace: exec: git-credential-manager store
  23:25:17.766671 run-command.c:663       trace: run_command: git-credential-manager store
- **** process git-credential-manager hangs ****
- **** process git-credential-manager killed, and then it logging further **** 
  23:26:39.343671 run-command.c:663       trace: run_command: 'git credential-wincred store'
  23:26:39.455671 exec-cmd.c:237          trace: resolved executable dir: C:/Dev/Git/mingw64/libexec/git-core
  23:26:39.566671 git.c:702               trace: exec: git-credential-wincred store
  23:26:39.566671 run-command.c:663       trace: run_command: git-credential-wincred store
  ...

Sometimes git-credential-manager get produces nothing, sometimes it results to one line only before it hangs:

... run_command: git-credential-manager get
23:28:54.019671 ...\Common.cs:744       trace: [Main] git-credential-manager (v1.20.0) 'get'
^C

Call-stacks of the hanging process It has no busy-wait, looks stable (not changing in 5 minutes), like a dead-lock.

1. Thread (2392) ``` ntdll.dll!ZwWaitForSingleObject+0xa ntdll.dll!RtlImageDirectoryEntryToData+0x118 ntdll.dll!RtlEnterCriticalSection+0xd1 rpcrt4.dll!UuidCreate+0x1dc5 rpcrt4.dll!I_RpcBindingIsServerLocal+0x12619 advapi32.dll!RegEnumKeyExW+0x13a advapi32.dll!RegEnumKeyExW+0xbe rpcrt4.dll!RpcBindingFree+0x320 rpcrt4.dll!RpcAsyncRegisterInfo+0x10ff rpcrt4.dll!Ndr64AsyncClientCall+0x9da rpcrt4.dll!Ndr64AsyncClientCall+0xc9b rpcrt4.dll!NdrClientCall3+0xf5 advapi32.dll!LsaOpenPolicy+0xb9 advapi32.dll!LsaOpenPolicy+0x56 !RemoveShellExt+0x16b !RemoveShellExt+0x3c7 !InstallSysMenuHook+0x26c8 ntdll.dll!RtlDeactivateActivationContextUnsafeFast+0x34e ntdll.dll!EtwDeliverDataBlock+0xa44 ntdll.dll!LdrLoadDll+0xed KernelBase.dll!LoadLibraryExW+0xea user32.dll!PostThreadMessageW+0xe1 ntdll.dll!KiUserCallbackDispatcher+0x1f user32.dll!SendMessageTimeoutW+0x95a user32.dll!SendMessageTimeoutW+0x8d7 user32.dll!SendMessageTimeoutW+0xb18 user32.dll!CreateWindowExW+0x70 ole32.dll!StringFromGUID2+0x2952 ole32.dll!StringFromGUID2+0x289d ole32.dll!CoInitializeEx+0x178 !InstallCustomModule+0x3e6c8 !SetRuntimeInfo+0x1be4 !PreBindAssemblyEx+0x1a80 !SetRuntimeInfo+0xa83 !SetRuntimeInfo+0xa01 !CorExeMain+0x14 !CorExeMain+0x71 !CorExeMain+0x69 kernel32.dll!BaseThreadInitThunk+0xd ntdll.dll!RtlUserThreadStart+0x1d ```
2. Thread (7956) ``` ntoskrnl.exe!KiCpuId+0xaa ntoskrnl.exe!ObfReferenceObject+0x792 ntoskrnl.exe!KeWaitForMutexObject+0x1a3 ntoskrnl.exe!CcGetFlushedValidData+0xc34 ntoskrnl.exe!KeInsertQueueApc+0x605 ntoskrnl.exe!ObfReferenceObject+0x99d ntoskrnl.exe!KeWaitForMultipleObjects+0x272 ntoskrnl.exe!NtRequestWaitReplyPort+0x335 ntoskrnl.exe!FsRtlReleaseFile+0xc6d ntoskrnl.exe!longjmp+0x5bf3 ntdll.dll!NtWaitForMultipleObjects+0xa KernelBase.dll!GetCurrentProcess+0x40 kernel32.dll!WaitForMultipleObjectsEx+0xb3 !GC_Initialize+0x1a7f !GC_Initialize+0x199b !GC_Initialize+0x1891 kernel32.dll!BaseThreadInitThunk+0xd ntdll.dll!RtlUserThreadStart+0x1d ```
3. Thread (9252) ``` ntoskrnl.exe!KiCpuId+0xaa ntoskrnl.exe!ObfReferenceObject+0x792 ntoskrnl.exe!KeWaitForMutexObject+0x1a3 ntoskrnl.exe!CcGetFlushedValidData+0xc34 ntoskrnl.exe!KeInsertQueueApc+0x605 ntoskrnl.exe!ObfReferenceObject+0x99d ntoskrnl.exe!KeWaitForMutexObject+0x1a3 ntoskrnl.exe!NtWaitForSingleObject+0xde ntoskrnl.exe!longjmp+0x5bf3 ntdll.dll!ZwWaitForSingleObject+0xa ntdll.dll!RtlImageDirectoryEntryToData+0x118 ntdll.dll!RtlEnterCriticalSection+0xd1 ntdll.dll!LdrQueryModuleServiceTags+0x13f ntdll.dll!CsrIdentifyAlertableThread+0x9d ntdll.dll!EtwSendNotification+0x16d ntdll.dll!RtlQueryProcessDebugInformation+0x371 ntdll.dll!EtwDeliverDataBlock+0xf00 kernel32.dll!BaseThreadInitThunk+0xd ntdll.dll!RtlUserThreadStart+0x1d ```
4. Thread (10776) ``` ntdll.dll!ZwWaitForSingleObject+0xa ntdll.dll!RtlImageDirectoryEntryToData+0x118 ntdll.dll!RtlEnterCriticalSection+0xd1 ntdll.dll!EtwDeliverDataBlock+0x777 ntdll.dll!LdrLoadDll+0xed KernelBase.dll!LoadLibraryExW+0xea rpcrt4.dll!UuidCreate+0x1de3 rpcrt4.dll!I_RpcBindingIsServerLocal+0x12619 advapi32.dll!RegEnumKeyExW+0x13a advapi32.dll!RegEnumKeyExW+0xbe rpcrt4.dll!RpcBindingFree+0x320 rpcrt4.dll!RpcAsyncRegisterInfo+0x10ff rpcrt4.dll!Ndr64AsyncClientCall+0x9da rpcrt4.dll!Ndr64AsyncClientCall+0xc9b rpcrt4.dll!NdrClientCall3+0xf5 advapi32.dll!LsaOpenPolicy+0xb9 advapi32.dll!LsaOpenPolicy+0x56 advapi32.dll!ConvertStringSidToSidW+0x276 advapi32.dll!ConvertStringSecurityDescriptorToSecurityDescriptorW+0xa32 advapi32.dll!ConvertStringSecurityDescriptorToSecurityDescriptorW+0x3a !IsOS+0x20f8 !IsOS+0x1f49 !CorExeMain+0x3255 !PreBindAssemblyEx+0x1434 !CoUninitializeEE+0x68c5 kernel32.dll!BaseThreadInitThunk+0xd ntdll.dll!RtlUserThreadStart+0x1d ```

Related issue I thought it could be similar #703, but it looks different (trace show Common.cs:744 only or even nothing and then hangs immediately).

sebres commented 5 years ago

Hmm... seems to be repaired from himself (anyway not reproducible anymore on my side). Since the proxy was involved in push-process and I don't know it had some troubles this 2 days or whether it could possibly bother here previously, so I'll close the issue. Feel free to reopen if you can exclude proxy problems...

sebres commented 5 years ago

This is hanging again today (with get)... Here are the call stacks of all 4 threads if it hangs (in "dead-lock")...

1. Thread (1184) ``` ntdll.dll!ZwWaitForSingleObject+0xa ntdll.dll!RtlImageDirectoryEntryToData+0x118 ntdll.dll!RtlEnterCriticalSection+0xd1 rpcrt4.dll!UuidCreate+0x1dc5 rpcrt4.dll!I_RpcBindingIsServerLocal+0x12619 advapi32.dll!RegEnumKeyExW+0x13a advapi32.dll!RegEnumKeyExW+0xbe rpcrt4.dll!RpcBindingFree+0x320 rpcrt4.dll!RpcAsyncRegisterInfo+0x10ff rpcrt4.dll!Ndr64AsyncClientCall+0x9da rpcrt4.dll!Ndr64AsyncClientCall+0xc9b rpcrt4.dll!NdrClientCall3+0xf5 advapi32.dll!LsaOpenPolicy+0xb9 advapi32.dll!LsaOpenPolicy+0x56 !RemoveShellExt+0x16b !RemoveShellExt+0x3c7 !InstallSysMenuHook+0x26c8 ntdll.dll!RtlDeactivateActivationContextUnsafeFast+0x34e ntdll.dll!EtwDeliverDataBlock+0xa44 ntdll.dll!LdrLoadDll+0xed KernelBase.dll!LoadLibraryExW+0xea user32.dll!PostThreadMessageW+0xe1 ntdll.dll!KiUserCallbackDispatcher+0x1f user32.dll!SendMessageTimeoutW+0x95a user32.dll!SendMessageTimeoutW+0x8d7 user32.dll!SendMessageTimeoutW+0xb18 user32.dll!CreateWindowExW+0x70 ole32.dll!StringFromGUID2+0x2952 ole32.dll!StringFromGUID2+0x289d ole32.dll!CoInitializeEx+0x178 !InstallCustomModule+0x3e6c8 !SetRuntimeInfo+0x1be4 !PreBindAssemblyEx+0x1a80 !SetRuntimeInfo+0xa83 !SetRuntimeInfo+0xa01 !CorExeMain+0x14 !CorExeMain+0x71 !CorExeMain+0x69 kernel32.dll!BaseThreadInitThunk+0xd ntdll.dll!RtlUserThreadStart+0x1d ```
2. Thread (7960) ``` ntoskrnl.exe!KiCpuId+0xaa ntoskrnl.exe!ObfReferenceObject+0x792 ntoskrnl.exe!KeWaitForMutexObject+0x1a3 ntoskrnl.exe!CcGetFlushedValidData+0xc34 ntoskrnl.exe!KeInsertQueueApc+0x605 ntoskrnl.exe!ObfReferenceObject+0x99d ntoskrnl.exe!KeWaitForMultipleObjects+0x272 ntoskrnl.exe!NtRequestWaitReplyPort+0x335 ntoskrnl.exe!FsRtlReleaseFile+0xc6d ntoskrnl.exe!longjmp+0x5bf3 ntdll.dll!NtWaitForMultipleObjects+0xa KernelBase.dll!GetCurrentProcess+0x40 kernel32.dll!WaitForMultipleObjectsEx+0xb3 !GC_Initialize+0x1a7f !GC_Initialize+0x199b !GC_Initialize+0x1891 kernel32.dll!BaseThreadInitThunk+0xd ntdll.dll!RtlUserThreadStart+0x1d ```
3. Thread (8952) ``` ntdll.dll!ZwWaitForSingleObject+0xa ntdll.dll!RtlImageDirectoryEntryToData+0x118 ntdll.dll!RtlEnterCriticalSection+0xd1 ntdll.dll!EtwDeliverDataBlock+0x777 ntdll.dll!LdrLoadDll+0xed KernelBase.dll!LoadLibraryExW+0xea rpcrt4.dll!UuidCreate+0x1de3 rpcrt4.dll!I_RpcBindingIsServerLocal+0x12619 advapi32.dll!RegEnumKeyExW+0x13a advapi32.dll!RegEnumKeyExW+0xbe rpcrt4.dll!RpcBindingFree+0x320 rpcrt4.dll!RpcAsyncRegisterInfo+0x10ff rpcrt4.dll!Ndr64AsyncClientCall+0x9da rpcrt4.dll!Ndr64AsyncClientCall+0xc9b rpcrt4.dll!NdrClientCall3+0xf5 advapi32.dll!LsaOpenPolicy+0xb9 advapi32.dll!LsaOpenPolicy+0x56 advapi32.dll!ConvertStringSidToSidW+0x276 advapi32.dll!ConvertStringSecurityDescriptorToSecurityDescriptorW+0xa32 advapi32.dll!ConvertStringSecurityDescriptorToSecurityDescriptorW+0x3a !IsOS+0x20f8 !IsOS+0x1f49 !CorExeMain+0x3255 !PreBindAssemblyEx+0x1434 !CoUninitializeEE+0x68c5 kernel32.dll!BaseThreadInitThunk+0xd ntdll.dll!RtlUserThreadStart+0x1d ```
4. Thread (10152) ``` ntoskrnl.exe!KiCpuId+0xaa ntoskrnl.exe!ObfReferenceObject+0x792 ntoskrnl.exe!KeWaitForMutexObject+0x1a3 ntoskrnl.exe!CcGetFlushedValidData+0xc34 ntoskrnl.exe!KeInsertQueueApc+0x605 ntoskrnl.exe!ObfReferenceObject+0x99d ntoskrnl.exe!KeWaitForMutexObject+0x1a3 ntoskrnl.exe!NtWaitForSingleObject+0xde ntoskrnl.exe!longjmp+0x5bf3 ntdll.dll!ZwWaitForSingleObject+0xa ntdll.dll!RtlImageDirectoryEntryToData+0x118 ntdll.dll!RtlEnterCriticalSection+0xd1 ntdll.dll!LdrQueryModuleServiceTags+0x13f ntdll.dll!CsrIdentifyAlertableThread+0x9d ntdll.dll!EtwSendNotification+0x16d ntdll.dll!RtlQueryProcessDebugInformation+0x371 ntdll.dll!EtwDeliverDataBlock+0xf00 kernel32.dll!BaseThreadInitThunk+0xd ntdll.dll!RtlUserThreadStart+0x1d ```
soyetola-atos commented 4 years ago

If this problem comes on a Windows machine, do the following.

For more options, check out the full thread on https://stackoverflow.com/questions/15381198/remove-credentials-from-git