cmd/dist: fails with buildvcs=false errors if invalid .git exists in parent of GOROOT_BOOTSTRAP

[paralin]

Related: #54852 - cc @williamh @bcmills

In Buildroot we have a build failure when there exists an invalid .git in a parent directory of GOROOT_BOOTSTRAP: https://gitlab.com/buildroot.org/buildroot/-/jobs/4725186525

Building Go toolchain1 using /builds/buildroot.org/buildroot/test-output/TestMender/host/lib/go-1.19.10.
error obtaining VCS status: exit status 128
    Use -buildvcs=false to disable VCS stamping.
error obtaining VCS status: exit status 128
    Use -buildvcs=false to disable VCS stamping.
error obtaining VCS status: exit status 128
    Use -buildvcs=false to disable VCS stamping.
error obtaining VCS status: exit status 128
    Use -buildvcs=false to disable VCS stamping.
go tool dist: FAILED: /builds/buildroot.org/buildroot/test-output/TestMender/host/lib/go-1.19.10/bin/go install -tags=math_big_pure_go compiler_bootstrap purego bootstrap/cmd/...: exit status 1

Reproduction of the issue:

# This reproduction sets up a directory structure that breaks Go make.bash.
# make.bash should not break due to .git files outside of the Go sources.
#
# The error specifically occurs when an invalid .git is present in a parent git
# directory of the GOROOT_BOOTSTRAP. Go issue #61620
mkdir go-issue-61620
cd ./go-issue-61620
wget https://go.dev/dl/go1.19.11.src.tar.gz
mkdir go-bootstrap
tar -xf go1.19.11.src.tar.gz -C ./go-bootstrap --strip-components=1
cd ./go-bootstrap/src/
bash make.bash
cd ../../
wget https://go.dev/dl/go1.20.6.src.tar.gz
mkdir go
tar -xf go1.20.6.src.tar.gz -C ./go/ --strip-components=1
printf "gitdir: ../../does/not/exist/.git" > ./.git
cd ./go/src/
GOROOT_BOOTSTRAP=$(pwd)/../../go-bootstrap/ bash make.bash
# Build fails using -buildvcs=false errors.

The error only occurs when the .git that git detects in the parent directory of the GOROOT_BOOTSTRAP is invalid or not present causing errors when running git commands within GOROOT_BOOTSTRAP.

[gopherbot]

Change https://go.dev/cl/513835 mentions this issue: cmd/dist: set buildvcs=false if not within a git repository

[dmitshur]

What is a sequence of steps that reproduces this issue in the Go project?

The following sequence does not produce an error:

$ cd $(mktemp -d)
tmp.tQscHcxE $ git clone https://go.googlesource.com/go
Cloning into 'go'...
[...]
tmp.tQscHcxE $ cd go
go $ rm -rf .git
go $ git status
fatal: not a git repository (or any of the parent directories): .git
go $ cd src
src $ ./make.bash
Building Go cmd/dist using /usr/local/go. (go1.21rc3 darwin/arm64)
go tool dist: FAILED: not a Git repo; must put a VERSION file in $GOROOT
src $ echo go1.22-issue61620 > ../VERSION 
src $ ./make.bash                        
Building Go cmd/dist using /usr/local/go. (go1.21rc3 darwin/arm64)
Building Go toolchain1 using /usr/local/go.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/arm64.
---
Installed Go for darwin/arm64 in /tmp/tmp.tQscHcxE/go
Installed commands in /tmp/tmp.tQscHcxE/go/bin
*** You need to add /tmp/tmp.tQscHcxE/go/bin to your PATH.
src $ echo $?
0
src $ export PATH="/tmp/tmp.tQscHcxE/go/bin:$PATH"
src $ go version
go version go1.22-issue61620 darwin/arm64

[paralin]

@dmitshur Reproduction of the issue:

# This reproduction sets up a directory structure that breaks Go make.bash.
# make.bash should not break due to .git files outside of the Go sources.
#
# The error specifically occurs when an invalid .git is present in a parent git
# directory of the GOROOT_BOOTSTRAP. Go issue #61620
mkdir go-issue-61620
cd ./go-issue-61620
wget https://go.dev/dl/go1.19.11.src.tar.gz
mkdir go-bootstrap
tar -xf go1.19.11.src.tar.gz -C ./go-bootstrap --strip-components=1
cd ./go-bootstrap/src/
bash make.bash
cd ../../
wget https://go.dev/dl/go1.20.6.src.tar.gz
mkdir go
tar -xf go1.20.6.src.tar.gz -C ./go/ --strip-components=1
printf "gitdir: ../../does/not/exist/.git" > ./.git
cd ./go/src/
GOROOT_BOOTSTRAP=$(pwd)/../../go-bootstrap/ bash make.bash
# Build fails using -buildvcs=false errors.

It is in itself a problem that the presence of .git directories in the parent folders above the Go sources have any influence on the behavior of make.bash.

The error only occurs when the .git that git detects in the parent directory of the GOROOT_BOOTSTRAP is invalid or not present causing errors when running git commands within GOROOT_BOOTSTRAP.

[paralin]

Reproduction using go master (go1.22) which requires go1.20 as go-bootstrap:

mkdir go-issue-61620
cd ./go-issue-61620
wget https://go.dev/dl/go1.20.6.src.tar.gz
mkdir go-bootstrap
tar -xf go1.20.6.src.tar.gz -C ./go-bootstrap --strip-components=1
cd ./go-bootstrap/src/
bash make.bash
cd ../../
git clone https://go.googlesource.com/go
rm -rf ./go/.git/
printf "gitdir: ../../does/not/exist/.git" > ./.git
printf 'go1.22.1' > ./go/VERSION
cd ./go/src/
GOROOT_BOOTSTRAP=$(pwd)/../../go-bootstrap/ bash make.bash
# Build fails using -buildvcs=false errors.

[bcmills]

https://go.dev/cl/432435 was intended to take care of exactly this problem. I suspect that the problem is that cfg.GOROOT is set to a cleaned path, whereas in this reproducer GOROOT_BOOTSTRAP is explicitly set to a non-clean path.

That said, now that the minimum bootstrap Go version is 1.20 (and we can therefore assume that the -buildvcs flag exists), I agree that it is cleaner to set -buildvcs explicitly.

[paralin]

@bcmills It was intended to take care of it, and did successfully, however as of Go 1.20 it's not working again (thus my PR).