git-for-windows / git

A fork of Git containing Windows-specific patches.
http://gitforwindows.org/
Other
8.38k stars 2.54k forks source link

Error when trying to fetch, clone, pull, push, or "remote show origin" #1742

Closed DeeDeeG closed 5 years ago

DeeDeeG commented 6 years ago

Setup

git version 2.18.0.windows.1 cpu: x86_64 built from commit: cd1a74fc9dc8a07626c216940db9a51f25206e03 sizeof-long: 4

(I am running the 64-bit copy of Git for Windows.)

 - Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 6.0.6002]

(Windows Vista SP2 64-bit, all Windows Updates installed)

 - What options did you set as part of the installation? Or did you choose the
   defaults?

$ cat /etc/install-options.txt

Editor Option: VIM Path Option: Cmd SSH Option: OpenSSH CURL Option: OpenSSL CRLF Option: CRLFAlways Bash Terminal Option: MinTTY Performance Tweaks FSCache: Enabled Use Credential Manager: Enabled Enable Symlinks: Disabled

(Deliberately installed with default options; Pressed only "Next" and "Finish" during install.)

 - Any other interesting things about your environment that might be related
   to the issue you're seeing?

Not sure? Other than the fact that Vista is pretty old.

### Details

 - Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

The issue happens with powershell, cmd.exe, or "Git CMD"; Does not happen with "Git Bash" or "Git GUI".

 - What commands did you run to trigger this issue? If you can provide a
   [Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve)
   this will help us understand the issue.

```bash
# [any valid clone command here, can target a truly "remote" remote, or a local remote, eg:] 
git clone https://github.com/git/git
git clone [local-repo-name] [local-repo-name-copy]

# Inside a git repo folder, any of the following (or valid variations thereof):
git pull
git fetch
git push
# or
git remote show origin

I expected these commands to complete without returning errors.

When cloning from a local git folder to another local git folder:

C:\Users\[my-username]>git clone daala daala-copy

Cloning into 'daala-copy'...
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

C:\Users\[my-username]>fatal: packet write with format failed: Bad file descriptor 

When cloning from a web remote to a local git folder:

C:\Users\[my-username]>git clone https://github.com/git/git
Cloning into 'git'...
fatal: Full write to remote helper failed: Broken pipe

C:\Users\[my-username]>error: remote-curl: error reading command stream from git

(Exact same error messages as above when running git fetch, git pull, git push, or git remote show origin. Happens regardless of whether these commands are run from a local git folder with a web remote, or a local git folder with another local git folder as its "origin" remote.)

The issue is not limited to any specific repository.

DeeDeeG commented 6 years ago

This happened for the first time when I had a couple of custom options selected, Particularly: always use a TrueType font, symbolic links enabled, no context menu integration, and nano as my default editor.

Then I completely uninstalled, then reinstalled fresh (using only the default install options) to see if that would solve my issue. (Since that did not solve my issue, I decided to go ahead and report the issue here.)

nathanwonnacott commented 6 years ago

I'm seeing the same issue with 2.10.1.windows.1, only for me I observe the issue in git-bash as well (and possibly in Git GUI too, but I don't know how to do anything in Git GUI, so I didn't test).

DeeDeeG commented 6 years ago

Hmm... @nathanwonnacott are you by chance seeing this on Windows Vista 64-bit, like I am?

dscho commented 5 years ago

I do not have access to Vista, unfortunately, so I cannot test this. Can you verify that this still happens with v2.21.0?

DeeDeeG commented 5 years ago

I can try it again in the next couple of days.

(Would be great also if someone on a different machine with Vista can confirm the bug!)

DeeDeeG commented 5 years ago

Looks like this still happens with Git for Windows 2.21.0.

I am on a fresh install of Vista 64-bit SP1 testing it out. Issues with Git CMD are exactly as described in the original comment of this issue, though I am now using the latest stable release of Git for Windows.

Git Bash and Git GUI seem to work flawlessly, which is also consistent with the original comment of this issue.

Will update my installation of Vista to SP2 and make sure all Windows updates are installed, then try it again.

dscho commented 5 years ago

I have an idea that it might be caused by https://github.com/git-for-windows/git/pull/1900... we thought that we removed some code necessary only with XP and older...

Could you test that hypothesis by trying with (portable) Git for Windows v2.19.1? That should be the latest version that works, if my hunch is correct.

nathanwonnacott commented 5 years ago

Sorry I've been neglecting to reply on this thread for so long. I experienced the issue at work and I don't have access to my work computers right now, so I'll try to check on this next time I'm in. I know that we were not using Vista though. I remember experiencing this issue at about the time that we were transitioning everything from Windows 7 to Windows 10. I don't recall if we saw this on Windows 7 computers or Windows 10 computers. We were able to fix the solution by installing a newer version of git though.

DeeDeeG commented 5 years ago

I'm on Vista SP2 now, all security uodates installed (optional updates not installed).

2.19.1 portable works identically to 2.21.0 (portable or installed) with regard to this bug.

(I recall that the the original comment of this issue reported the problem happening with 2.18.0 as well. I guess I could go back and try to "bisect" the releases and see when it stops working... Which would take a fair amount of time. Will look for easier solutions first.)

DeeDeeG commented 5 years ago

@nathanwonnacott, I noticed there is a minor typo at the end of your comment. Did you mean you needed an updated git? Maybe curl?

DeeDeeG commented 5 years ago

Okay, I went and bisected this (in terms of release milestones, not as deep as commits yet...)

Full bisection notes (tap/click/press "Enter" to expand) (when not stated otherwise, version under test was the "[version].windows.1" release. All testing done with Git for Windows portable only, Git CMD only.) Bisecting 2.5 (CMD works) to 2.21 (CMD is faulty) 2.5.0: all good 2.13: all good 2.17: issues with Git CMD Bisecting 2.13 to 2.17 2.15: all good [No 2.16.0.windows.1 available to test, moving on to the next closest version.] 2.16.0.windows.2: all good 2.16.2: issues with Git CMD Bisecting 2.16.0.2 to 2.16.2 2.16.1.windows.1: all good 2.16.1.windows.3: issues with Git CMD One version left to test between (2.16.1) .windows.1 and .windows.3... 2.16.1.windows.2: issues with CMD

The problem seems to have been introduced sometime between 2.16.1.windows.1 and 2.16.1.windows.2.

dscho commented 5 years ago

The problem seems to have been introduced sometime between 2.16.1.windows.1 and 2.16.1.windows.2.

There have been quite a few package updates between those versions:

diff --git a/versions/package-versions-2.16.1.txt b/versions/package-versions-2.16.1.2.txt
index 1f4cce4..d3ed9c7 100644
--- a/versions/package-versions-2.16.1.txt
+++ b/versions/package-versions-2.16.1.2.txt
@@ -10,30 +10,27 @@ dos2unix 7.4.0-1
 expat 2.2.5-1
 file 5.32-1
 findutils 4.6.0-1
-gawk 4.1.4-2
-gcc-libs 6.4.0-1
-gdbm 1.11-3
+gawk 4.2.0-1
+gcc-libs 6.4.0-2
 gettext 0.19.8.1-1
-git-extra 1.1.225.cf6c25d-1
+git-extra 1.1.239.b06544d-1
 git-flow 1.11.0-1
 glib2 2.48.2-1
 gmp 6.1.2-1
 gnupg 1.4.22-2
 grep 3.0-1
 gzip 1.9-1
-heimdal-libs 1.5.3-9
+heimdal-libs 7.5.0-1
 icu 59.1-1
 less 529-1
 libasprintf 0.19.8.1-1
 libbz2 1.0.6-2
 libcrypt 2.1-1
-libcurl 7.57.0-2
-libdb 5.3.28-2
+libcurl 7.58.0-1
 libedit 3.1-20170329
 libexpat 2.2.5-1
 libffi 3.2.1-1
 libgcrypt 1.8.1-1
-libgdbm 1.11-3
 libgettextpo 0.19.8.1-1
 libgpg-error 1.27-1
 libiconv 1.14-2
@@ -45,6 +42,7 @@ libnghttp2 1.24.0-1
 libopenssl 1.0.2.n-5
 libp11-kit 0.23.9-1
 libpcre 8.40-2
+libpsl 0.18.0-2
 libreadline 7.0.003-1
 libsasl 2.1.26-8
 libserf 1.3.9-1
@@ -58,15 +56,15 @@ libxslt 1.1.32-1
 mingw-w64-x86_64-antiword 0.37-2
 mingw-w64-x86_64-bzip2 1.0.6-6
 mingw-w64-x86_64-ca-certificates 20170211-2
-mingw-w64-x86_64-c-ares 1.13.0-3
+mingw-w64-x86_64-c-ares 1.13.0-4
 mingw-w64-x86_64-connect 1.104-1
-mingw-w64-x86_64-curl 7.57.0-2
+mingw-w64-x86_64-curl 7.58.0-1
 mingw-w64-x86_64-expat 2.2.5-1
-mingw-w64-x86_64-gcc-libs 7.2.0-2
+mingw-w64-x86_64-gcc-libs 7.3.0-1
 mingw-w64-x86_64-gettext 0.19.8.1-2
-mingw-w64-x86_64-git 2.16.1.1.1a4ee4d5d8-1
+mingw-w64-x86_64-git 2.16.1.2.e78e3c8ee9-1
 mingw-w64-x86_64-git-credential-manager 1.14.0-1
-mingw-w64-x86_64-git-doc-html 2.16.1.1.1a4ee4d5d8-1
+mingw-w64-x86_64-git-doc-html 2.16.1.2.e78e3c8ee9-1
 mingw-w64-x86_64-git-lfs 2.3.4-1
 mingw-w64-x86_64-gmp 6.1.2-1
 mingw-w64-x86_64-jansson 2.10-1
@@ -75,32 +73,31 @@ mingw-w64-x86_64-libiconv 1.15-1
 mingw-w64-x86_64-libidn2 2.0.4-1
 mingw-w64-x86_64-libmetalink 0.1.3-3
 mingw-w64-x86_64-libssh2 1.8.0-1
-mingw-w64-x86_64-libtasn1 4.12-1
+mingw-w64-x86_64-libtasn1 4.13-1
 mingw-w64-x86_64-libunistring 0.9.8-1
-mingw-w64-x86_64-libwinpthread-git 5.0.0.4850.d1662dc7-1
+mingw-w64-x86_64-libwinpthread-git 6.0.0.5098.2464b7e6-1
 mingw-w64-x86_64-libzip 1.3.0-1
 mingw-w64-x86_64-mpc 1.1.0-1
 mingw-w64-x86_64-mpfr 4.0.0-1
-mingw-w64-x86_64-nghttp2 1.26.0-1
+mingw-w64-x86_64-nghttp2 1.29.0-1
 mingw-w64-x86_64-odt2txt 0.5-2
 mingw-w64-x86_64-openssl 1.0.2.n-5
 mingw-w64-x86_64-pcre 8.41-1
-mingw-w64-x86_64-spdylay 1.4.0-1
 mingw-w64-x86_64-tcl 8.6.8-1
 mingw-w64-x86_64-tk 8.6.8-1
 mingw-w64-x86_64-wineditline 2.201-1
 mingw-w64-x86_64-wintoast 1.0.0.181.9b0663d-1
 mingw-w64-x86_64-zlib 1.2.11-1
 mintty 1~2.8.3-1
-mpfr 3.1.5.1-3
+mpfr 4.0.0.1-3
 msys2-runtime 2.9.0-7
 nano 2.8.7-1
-ncurses 6.0.20170708-2
+ncurses 6.1.20180127-1
 openssh 7.6p1-1
 openssl 1.0.2.n-5
 p11-kit 0.23.9-1
 patch 2.7.5-1
-perl 5.24.1-3
+perl 5.26.1-1
 perl-Authen-SASL 2.16-2
 perl-Convert-BinHex 1.123-2
 perl-Encode-Locale 1.04-1
@@ -130,7 +127,7 @@ sed 4.4-2
 ssh-pageant-git 1.4.12.g6f47092-1
 subversion 1.9.7-1
 tar 1.30-1
-tig 2.3.0-1
+tig 2.3.3-1
 unzip 6.0-2
 vim 8.0.1305-1
 which 2.21-2

As to Git itself, it also saw (very few) changes:

$ git log --oneline --boundary --graph v2.16.1.windows.1..v2.16.1.windows.2                                                                                       
* e78e3c8ee9 squash! mingw: spawned processes need to inherit only standard handles
*   3016f0c4bc Merge pull request #1450 from shiftkey/schannel-norevoke-support
|\  
| * 3fdedea6b8 add support for disabling SSL revocation checks in cURL
* |   576ff26eec Merge branch 'inherit-only-stdhandles'
|\ \  
| |/  
|/|   
| * cf2f73537c mingw: spawned processes need to inherit only standard handles
| * 31a5d68e4e mingw: bump the minimum Windows version to Vista
| * 5d8647e3b6 mingw: set _WIN32_WINNT explicitly for Git for Windows
| * 6ea71ff39a compat/poll: prepare for targeting Windows Vista
| * d600862a04 mingw: demonstrate that all file handles are inherited by child processes
|/  
o 1a4ee4d5d8 fixup! checkout.c: enable fscache for checkout_entry

It could be the changes in git.exe itself (maybe you want to unpack the portable Git versions, and copy over mingw64\bin\git.exe from the working to the non-working one and test?

It could also be a package update that broke things, since you reported network-related issues, what sticks out is mingw-w64-curl (mingw64\bin\libcurl4.dll and mingw64\libexec\git-core\libcurl4.dll) and mingw-w64-c-ares (mingw64\bin\libcares-3.dll and mingw64\libexec\git-core\libcares-3.dll), if copying the working git.exe does not fix things, maybe copying one or both of those .dll files does?

dscho commented 5 years ago
-mingw-w64-x86_64-nghttp2 1.26.0-1
+mingw-w64-x86_64-nghttp2 1.29.0-1

It could also be that, of course, which translates to the libnghttp2-14.dll file in those two directories.

nathanwonnacott commented 5 years ago

@DeeDeeG thanks for the correction. I don't think my wife would be too happy if I installed a newer version of "girls". I did mean "git". Sorry, typing on a phone.

DeeDeeG commented 5 years ago

I'm not sure if this is progress or one step back one step forward. Anyway, it's more info, so I'll post it here:

Test results from replacing components (click/tap/Enter key to expand) These tests involve transplanting components from the working versio (2.16.1.windows.1) into the non-working version (2.16.1.windows.2) Test is then run with the modified 2.16.1.windows.2 Git CMD The test itself is attempting to clone a repository from Github. Results have the format: replaced: [path]\file (whereas [path] is understood to be a subdirectory of mingw64, and "libexec\" is a shorthand for "libexec\git-core\".) results: [error message or component that crashed] --- replaced: (nothing) result: "fatal: Full write to remote helper failed: Broken pipe" error: cannot spawn git-remote-https: Invalid argument --- replaced: bin\git.exe result: crash in "git-remote-https.exe" replaced: bin\git.exe AND libexec\git-remote-https.exe result: "error: remote-curl: error reading comand stream from git" --- replaced: bin\libcurl-4.dll result: "fatal: Full write to remote helper failed: Broken pipe" "error: cannot spawn git-remote-https: Invalid argument" replaced: bin\libcurl4.dll AND libexec\libcurl-4.dll reult: "fatal: Full write to remote helper failed: Broken pipe" "error: cannot spawn git-remote-https: Invalid argument" --- replaced: bin\libcurl4.dll AND libexec\libcurl-4.dll AND bin\libcares-2.dll reult: "fatal: Full write to remote helper failed: Broken pipe" "error: cannot spawn git-remote-https: Invalid argument" replaced: bin\libcurl4.dll AND libexec\libcurl-4.dll AND bin\libcares-2.dll AND libexec\cares-2.dll reult: "fatal: Full write to remote helper failed: Broken pipe" "error: cannot spawn git-remote-https: Invalid argument" --- replaced: bin\libcares-2.dll AND libexec\cares-2.dll result: all good (clone works, as does "remote show origin" and "fetch")

So from this testing, I noticed that 2.16.1.windows.2 actually has a unique error message, even without any component swapping:

fatal: Full write to remote helper failed: Broken pipe
error: cannot spawn git-remote-https: Invalid argument

That can be fixed by swapping in libcares-2.dll, at which point 2.16.1.windows.2 works as expected.

The "one step back" is that this doesn't fix the current 2.21.0 release on Vista, so I'm not sure how universally applicable the fix is. That's what I have for now.

(I also tried swapping libnghttpd, which isn't in my notes above, but unless I messed up somehow, that didn't seem to change anything.)

DeeDeeG commented 5 years ago

I can get git fetch and gut remote show origin to work in 2.21.0 by swapping in mingw64/bin/git.exe and mingw64/libexec/git-core/git.exe from 2.16.1.windows.1

But git clone doesn't work still.

It gives this error message (when trying to clone https://github.com/git-for-windows/git):

Cloning into 'git'...
fatal: Not a git repository: C:/[path/including/current/working/directory]/git/.git

So it appears to try to create a repository locally in git/.git, but fails to finish doing so.

dscho commented 5 years ago

Indeed, this is puzzling, and looks like there are at least two independent bugs at play :-(

Just to make sure, v2.21.0 has the problem "error: cannot spawn git-remote-https: Invalid argument" when you do not replace git.exe in mingw64/bin/ and mingw64/libexec-git-core/?

If so, do you see that problem with other Git commands that spawn other Git commands, such as git gc (or git difftool when there are unstaged changes)?

DeeDeeG commented 5 years ago

Just to make sure, v2.21.0 has the problem "error: cannot spawn git-remote-https: Invalid argument" when you do not replace git.exe in mingw64/bin/ and mingw64/libexec-git-core/?

To clarify, the error in this situation (fresh Git for Windows v2.21.0, and trying to clone on Vista) is:

Cloning into 'git'...
fatal: Full write to remote helper failed: Broken pipe

which is the same error reported in the original comment of this issue.

For an older version, (freshly extracted 2.16.1.windows.2 on Vista, and trying to clone) this is the error:

fatal: Full write to remote helper failed: Broken pipe
error: cannot spawn git-remote-https: Invalid argument

I don't see this exact message anywhere other than 2.16.1.2, as far as I can recall.

DeeDeeG commented 5 years ago

And the older version (2.16.1.windows.2) can be fixed by swapping the previous release's libcares-2.dll files in mingw64/bin/ and mingw64/libexec/git-core/

("previous release" in this case referring to 2.16.1.windows.1)

Maybe I need to go ahead and ignore the breakage in 2.16.1.windows.2, since it seems to be unrelated to the breakage I'm seeing in 2.21, and 2.16.1.windows.2's breakage looks like it might have been fixed in subsequent releases.

I can try some more "bisecting" with the assumption that the 2.16.1.windows.2 breakage is an unrelated and already-solved bug. Should get closer to finding the cause of breakage in current releases.

DeeDeeG commented 5 years ago

I tried reproducing my "fix" involving libcares-2.dll, described above, and I can't.

From reading the release notes, there were a few issues that popped up here on GitHub between 2.16.1.1 and 2.17. I'm not sure if any of those existing issues are what I'm seeing. At the moment it seems like "one or two things between 2.16.1.1 and 2.17" is the problem.

Personally, I am able to use git bash. And I rarely need to use my Windows computer, which so happens to be running Vista, which happens to be "out of support" per Microsoft. So, I'm not sure how much more effort to put into this? I'll try to keep diagnosing if we have an obvious trail to check on. But I'm failing to see a single cause over a single release milestone.

demon-ru commented 5 years ago

I have the same error on Windows 2008 Server Standard SP2: fatal: full write to remote helper failed: Broken pipe error: remote-curl: error reading command stream from git

DeeDeeG commented 5 years ago

@demon-ru thanks for mentioning that you have this error too. I am finding it tricky to troubleshoot the exact cause of the problem. Any additional insights here would be appreciated, so that hopefully they can lead to a fix.

But for now, a workaround is to use "Git bash", or to install Git for Windows 2.16.1.windows.1.

dscho commented 5 years ago

@DeeDeeG if you get a chance to build Git for Windows yourself, you could try whether forcing restrict_handle_inheritance to 0 would work around things for you?

DeeDeeG commented 5 years ago

While I feel you conveyed this clearly, in case something odd happens it's good to be 100% sure:

I just need to switch 1 to 0 at that line of that file?

Should I prioritize trying this against the latest stable release tag? Or against master? (I will do both if need be.)

(And it looks like I can compile on Linux and then run on Windows? That's what I'll do, if possible.)

(Edit: It looks unlikely I can compile on Linux. Seems INSTALL.md is from upstream, whereas CONTRIBUTING.md is more specific to this project... I'll give it a shot!)

DeeDeeG commented 5 years ago

I'm running into trouble installing the Git for Windows sdk, since the installer seems to use Git CMD to clone Git-for-Windows. As you can understand, this isn't working on my Vista machine.

Will see if I can borrow a family member's computer and build it there.

DeeDeeG commented 5 years ago

Not sure how to copy the built Git over to another computer.

If you can advise what steps you would expect one to have to do to build on one computer and run the built Git on another computer, that would help me try this fix.

dscho commented 5 years ago

I just need to switch 1 to 0 at that line of that file?

Yes.

Should I prioritize trying this against the latest stable release tag? Or against master? (I will do both if need be.)

Better test against master.

Not sure how to copy the built Git over to another computer.

If you can advise what steps you would expect one to have to do to build on one computer and run the built Git on another computer, that would help me try this fix.

The easiest would be to build a portable Git, I think:

sdk cd build-extra
./portable/release.sh 0-test

It will tell you where it put the generated .7z.exe, which is a self-extracting 7-Zip archive. This can be copied to the other machine, be extracted e.g. onto the Desktop, and run from there.

Thank you for going through all that trouble to investigate further! I understand that it is quite a bit of work, and absent a way for me to reproduce the problem locally, there is unfortunately little I can do to help...

DeeDeeG commented 5 years ago

Thank you for going through all that trouble to investigate further!

Yes, I consider it a service to all the other hardy Vista users who may still be out there, navigating the post-Microsoft-support landscape... 👍

One more question, if I may ask: It’s unclear to me what source git tree (on my local disk) the sdk draws from, when I run that script to build the portable installer.

It didn’t seem to be in the build-extra directory anywhere I could notice... and it seems unlikely that it used my local copy of git with a non-default folder name (git-for-windows-source) in the C:\Users\[Me] folder.

I was able to generate the portable 7zip installer, but it just seems to need to copy raw files, not compile anything... How do I modify the Git source this process uses? (Or how do I point it to my modified Git source tree?)

DeeDeeG commented 5 years ago

Update: I think the build script for the portable installer just copies binaries and such from some locations, which are mostly shipped (pre-compiled) with the SDK.

To workaround the portable script not building Git from scratch, I will build a patched Git, and manually plunk the patched+compiled binaries where the "portable git" build script looks for them.

I can do this easily with a new git.exe (at /cmd/git.exe), but it's less clear if I need to make a new mingw64/share/git/compat-bash.exe or any of the deeper mingw64/[ . . .] files...


FWIW I was able to get the SDK set up on my Vista machine, by cloning https://github.com/git-for-windows/git-sdk-64/ using an older git-portable.

If there's a simpler way to test this, now that I don't necessarily need to build a new git-portable, I'm open to it.

dscho commented 5 years ago

Sorry, I wish there were more people answering these questions, people who do not do this for as long as I do it, and therefore are less prone to forget crucial parts.

To build a portable Git from a patched source, do

sdk cd git
<patch the source>
make -j$(nproc) install

# Now the binaries in the SDK are replaced by the just-built one;
# They will be a bit larger because they contain debug builds, but that is alright

sdk cd build-extra
./portable/release.sh 0-test

Sorry!

DeeDeeG commented 5 years ago

If you know whereabouts in the Wiki that should go, I can take a crack at adding those instructions to the Wiki.

EVUS2019 commented 5 years ago

DeeDeeG, thank you very, very much for your suggestion to install Git for Windows 2.16.1.windows.1 as a work around for Demon-Ru issue. I had the same issue while running git clone using DOS commands on a Windows 2008 server: fatal: full write to remote helper failed: Broken pipe error: remote-curl: error reading command stream from git I don't know if is important but the git config --list is also broken. It always returns "Missing filename ("less --help" for help)"

dscho commented 5 years ago

If you know whereabouts in the Wiki that should go, I can take a crack at adding those instructions to the Wiki.

Maybe add a new page about the portable Git flavor of Git for Windows? Or enhance https://github.com/git-for-windows/git/wiki/Making-an-installer? Dunno...

DeeDeeG commented 5 years ago

Okay, so I managed to build and make install the patched Git.

There is still a bug when using Git CMD; Regardless of which Git subcommand I do, or just git with no subcommand, I see this error message four times:

The code execution cannot proceed because [library] was not found. Reinstalling the program may fix this problem.

Where [library] is one of these, in sequence (random order, I think): libintl-8.dll libpcre2-8-0.dll zlib1.dll libiconv-2.dll

Maybe it's looking for these from my system instead of bundling them? I'm guessing I might not have done the right steps to include these libraries. (Still building and testing on Windows 10 at the moment, since the SDK is behaving a bit weirdly on Windows Vista.)

dscho commented 5 years ago

These libraries are in /mingw64/bin, so if you try to start git.exe directly in a CMD window in the directory where you built it, those libraries are most likely not in the PATH. If you use Git SDK Bash, they should be in the PATH, though...

DeeDeeG commented 5 years ago

It is still a bit unclear to me how to build a git.exe or a git CMD that runs regardless of the SDK being on my system. Or without those libraries at /mingw64/bin.

I believe this is needed for e.g. "Portable Git for Windows" to work.

PhilipOakley commented 5 years ago

You should look for details of "making an installer" (or something like that), which will generate, obviously, an installer that can be taken to whatever machine you want to try Git on, including a portable version. (sorry for the short answer - I'm away at the moment)

DeeDeeG commented 5 years ago

Another question, for curiosity's sake:

Should I update SDK packages with pacman? My instinct is to leave them at the version shipped with the SDK when I downloaded it, to reduce the chances that I personally caused a hard-to-track-down issue. Changing fewer variables is best for troubleshooting IMO.

dscho commented 5 years ago

Changing fewer variables is best for troubleshooting IMO.

Yep.

It is still a bit unclear to me how to build a git.exe or a git CMD that runs regardless of the SDK being on my system. Or without those libraries at /mingw64/bin.

I believe this is needed for e.g. "Portable Git for Windows" to work.

No, when you run /usr/src/build-extra/portable/release.sh 0-test, it will bundle enough of the files from the SDK that you do not need the SDK anymore.

However, before you build that portable Git, you definitely have to call make install in your Git worktree so as to install into /mingw64/bin/git.exe and /mingw64/libexec/git-core/ (and a couple of other places, too, I think).

DeeDeeG commented 5 years ago

I believe this is working(!!!!)


I had to nuke (delete) my SDK installation I had been using (on a Wndows 10 machine) and start fresh... Then I followed the exact instructions for patching and make installing git into my SDK folders. And then I exactly followed the steps to make a portable Git for Windows... Which I then copied over to Vista and extracted there.

This worked fine doing the following quick tests under Git CMD:

All of that worked fine.


Here is the exact patch to the Git tree that I used... https://github.com/DeeDeeG/git/commit/ae4432df4cb3b501476dcd895a955cf03d9276cd.patch (Just following @dscho's thing he wanted to test.)

DeeDeeG commented 5 years ago

Would be happy to submit the patch as a Pull Request, (or even just to get a mention in a commit message like "thanks @deedeeg for testing" or something like that).

I am pretty sure this fixes the problem on Vista. (Probably on Windows Server 2008 as well, since they're mostly the same thing.)

Let me know when you get the chance to see this; I'm eager to see Vista support restored to Git for Windows.

dscho commented 5 years ago

Would be happy to submit the patch as a Pull Request

Great!

Obviously, we cannot force it off by default. There are locking issues that are prevented by this "inherit standard handles only" code.

What we can do is to introduce a config setting, say, core.restrictInheritedHandles (modeled after core.longPaths/core_long_paths i.e. adding a global after core_long_paths, that defaults to -1 and handling the config parsing in mingw_core_config(), and then using it via int restrict_handle_inheritance = !!core_restrict_inherited_handles;.

And of course testing it 😸

It would be even neater if you could find out under what circumstances this fails, of course, and I guess if it turns out that it is simply the Windows version that describes those circumstances, we could use something like this to override the default for restrict_handle_inheritance if core_restrict_inherited_handles == -1:

        /*
         * On Windows 7 the following code to restrict which handles are inherited
         * does not work because [... fill in explanation here...]
         */
        int restrict_handle_inheritance = core_restrict_inherited_handles;
        if (restrict_handle_inheritance < 0)
            restrict_handle_inheritance = GetVersion() >> 16 >= 9200;
DeeDeeG commented 5 years ago

Thank you for the thorough reply!

I don't think I know enough C to understand how to implement this fix. (I've done beginner programming tutorials only.)

I've only been able to narrow it down to "all of Vista" (plus Server 2008, apparently). (I note that this reproduces on a clean re-install of Vista. But maybe it's something odd with my machine, my OEM-custom Vista install disc, my hardware/drivers... I can direct-message or e-mail you more specs and details about my machine if you like.)

I'm happy to test fixes in the hypothetical PR on my Vista machine, if that will help. Also, I'd be up for trying tweaking things on my Vista test environment (following specific suggestions) to rule things out, if I can.

I may be able to test on XP too, if Git will run there... but only if that seems helpful for narrowing down the source of the problem on Vista; I don't really strongly want to go down a rabbit hole of attempting to support XP again.

dscho commented 5 years ago

I may be able to test on XP too, if Git will run there...

No, it won't even compile there. I sent out multiple calls for action a couple of years, trying to encourage those who want XP support to work with me on reinstating it, but when there was no reaction whatsoever for one year, I figured that if I am the only one putting any kind of effort into this project, then it's probably not worth it. I, for one, don't need XP support so why should I care more than XP users? So XP support is dead as of Git for Windows v2.10.1, and I don't think that even a dedicated developer could bring it back by now, there has just been too much cleanup of all those XP-specific hacks.

I've only been able to narrow it down to "all of Vista" (plus Server 2008, apparently)

Do you mean build numbers lower than 7600? (I found https://www.lifewire.com/windows-version-numbers-2625171 and https://www.prajwaldesai.com/windows-operating-system-version-numbers/ which claim that Windows 7 is build number 7600, and Vista and 2008 are below, you should be able to confirm using ver in CMD).

I don't think I know enough C to understand how to implement this fix. (I've done beginner programming tutorials only.)

I hoped that my answer would provide enough detail to allow you to move this project forward, no matter what your C fu looks like :-(

I'm happy to test fixes in the hypothetical PR on my Vista machine, if that will help.

It will! Here is the PR: https://github.com/git-for-windows/git/pull/2210

Could you please build and test? If we can get this done soon enough, it might still make it into Git for Windows v2.22.0 which is due soon (originally targeted for Monday, it might slip for a day or three).

Thank you!

dscho commented 5 years ago

(I've done beginner programming tutorials only.)

This reminded me of something I read on Twitter recently, and truer words have rarely been spoken:

The first step in solving any problem is to dramatically underestimate its difficulty.

I would like to encourage you to follow that advice. If I had not dramatically, no, wait, DRAMATICALLY underestimated the difficulty of porting Git to Windows, you would not be using it today. I recently did a back-of-the-envelope estimate how many hours I poured into this project so far, and I think I came up with something like 20,000. My original estimate when I started was that it'd take me maybe four weeks.

All this is to say: you might currently think that your C fu is not up to snuff, but if you just ignore the anxiety and go with "how hard can it be, really", you'll be coding in C like a goddess in no time, contributing important features and bug fixes e.g. to Git.

DeeDeeG commented 5 years ago

Do you mean build numbers lower than 7600? [...] you should be able to confirm using ver in CMD).

My ver output is: Microsoft Windows [Version 6.0.6002]

[Edit to add: I just mean it only happens for me on Vista, so the best I can do for defining the boundaries of where this reproduces is “Vista (and someone else said Server 2008, which kinda makes sense).” But I can’t think of ways to narrow it down, so that’s the best I can do for now.]

Here is the PR: #2210

Will test soon!

[encouragement to try programming]

I know I'm jumping over most of your comment here, but consider it noted... I do do the occasional "tilting at windmills" style of learning programming, and I do find it teaches me a lot! Starting with the assumption that it can be done and going from there has been a great strategy!

[Edit: by “tilting at windmills,” I mean attacking an obstacle that seems to be logically/objectively/rationally insurmountable, but which one has the feeling one might be able to beat. Not that the obstacle is imaginary or actually unbeatable! I just like the Don Quixote reference and I think it makes a good metaphor.]

That said, these days I'm feeling more like learning Python or something gentle, high-level, interpreted, etc. before moving on to something more to do with memory management like C, Rust... C's syntax, and doing everything manually just puts me in a bit of a panic mode TBH! So yeah, perhaps I'll get there, but hacking Git seems a bit beyond me at the moment.

(With C: I've gotten up to making/using an additional function other than main(), and got stuck there. So I have a way to go before I'm zooming around multi-file code-bases, tracking variables... etc. I don't even know what I'm looking at sometimes with these large C projects! A lot of language features I can only guess how they work. But I'm still going to try to learn, whenever I get the time to!)

P.S. If someone here wants to coach me pretty much STEP BY STEP through doing this fix, I'd be willing/interested to try. I would have a lot of questions though. Expecting it would take a few days to a month, lol.

DeeDeeG commented 5 years ago

[Edit: I am compiling and packing up the "portable Git" for PR #2210. I haven't looked at the code there, and in the mean-time, I am thinking about how I would fix this myself.]

As for a fix: in pseudocode, I would do this:

IF vista or earlier, THEN [set variable to 0] ELSE [set variable to 1].

But...

Am I right to assume core.[something] as a "preference" is a Git configuration? as in git config core.editor?

if core_restrict_inherited_handles == -1

Where does -1 come from? So far I've been thinking in terms of 0=OFF, 1=ON. Unclear what system is introducing/using negative numbers.

restrict_handle_inheritance = GetVersion() >> 16 >= 9200

[Edit to add another question: What kind of values does GetVersion() usually return? It looks like this is the Windows version, and presumably has to do with build IDs you mentioned, such as 7600, 6002 etc. But I'm a little nervous about following the bit-shifting by 16 bits, and why the number your code snippet checks for is 9200, which seems pretty far from the dividing line I would expect between Vista and Windows 7 (I would expect the dividing line to be at around build ID ~7000).]

DeeDeeG commented 5 years ago

Not sure the merits for or against, but Microsoft documentation seems to recommend a different way of getting the system version: https://docs.microsoft.com/en-us/windows/desktop/sysinfo/getting-the-system-version

something like:

restrict_handle_inheritance = IsWindows7OrGreater()

Old API: https://docs.microsoft.com/en-us/windows/desktop/api/sysinfoapi/nf-sysinfoapi-getversion New helper functions from Windows 8.1 SDK: https://docs.microsoft.com/en-us/windows/desktop/SysInfo/version-helper-apis

DeeDeeG commented 5 years ago

A bit beyond strictly the scope of this bug report, but I am trying to follow the instructions in https://github.com/git-for-windows/git/issues/1742#issuecomment-497466021 for my own learning.

I am wondering whereabouts this should go:

        /*
         * On Windows 7 the following code to restrict which handles are inherited
         * does not work because [... fill in explanation here...]
         */
        int restrict_handle_inheritance = core_restrict_inherited_handles;
        if (restrict_handle_inheritance < 0)
            restrict_handle_inheritance = GetVersion() >> 16 >= 9200;

Inside a particular function? Right above/below a certain function? in place of this?:

int restrict_handle_inheritance = !!core_restrict_inherited_handles;

Any help understanding the problem will be appreciated, just to help pick up a bit of C skills. (But in any case, I'm glad PR #2210 exists, so Vista might be fully working again. soon!)

dscho commented 5 years ago

Not sure the merits for or against, but Microsoft documentation seems to recommend a different way of getting the system version: https://docs.microsoft.com/en-us/windows/desktop/sysinfo/getting-the-system-version

Right, I read that, but I also suspect that the new APIs do nothing else than call the old API, and the old API will stay supported for a long time, so...