search

weaveworks / scope

Monitoring, visualisation & management for Docker & Kubernetes
https://www.weave.works/oss/scope/
Apache License 2.0
5.84k stars 708 forks source link

Cannot build latest release (v1.13.1), cannot find package "github.com/k-sone/critbitgo" & dependency github.com/mjibson/appstats/ 410 Gone #3837

Closed dionysius closed 3 years ago

dionysius commented 3 years ago

What you expected to happen?

Building latest release (currently v1.13.1) succeeds.

What happened?

First, build fails because of an allegedly missing dependency. Quite questionable because go vendoring is used and vendor path exists. When trying fix it by letting go to reapply the vendor folder, that fails as well due to another dependency is gone. I found out that go modules is turned off in a specific command, so this repo now is expected to be within GOPATH or we turn go modules on. Either way would not solve the issue, the second dependency is still missing in vendor. So the issue here is, the provided vendor folder must be complete, but according to my investigation, it is not.

How to reproduce it?

diff --git a/Makefile b/Makefile
index 2a318409..90e3c41a 100644
--- a/Makefile
+++ b/Makefile
@@ -103,7 +103,7 @@ $(SCOPE_EXE):

 %.codecgen.go: $(CODECGEN_EXE)
        rm -f $@; $(GO_HOST) build $(GO_BUILD_FLAGS) ./$(@D) # workaround for https://github.com/ugorji/go/issues/145
-       cd $(@D) && $(WITH_GO_HOST_ENV) GO111MODULE=off $(shell pwd)/$(CODECGEN_EXE) -d $(CODECGEN_UID) -rt $(GO_BUILD_TAGS) -u -o $(@F) $(notdir $(call GET_CODECGEN_DEPS,$(@D)))
+       cd $(@D) && $(WITH_GO_HOST_ENV) $(shell pwd)/$(CODECGEN_EXE) -d $(CODECGEN_UID) -rt $(GO_BUILD_TAGS) -u -o $(@F) $(notdir $(call GET_CODECGEN_DEPS,$(@D)))

 $(CODECGEN_EXE): $(CODECGEN_DIR)/*.go
        mkdir -p $(@D)

Anything else we need to know?

Versions:

$ go version
go version go1.14.7 linux/amd64

$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/REDACTED/.cache/go-build"
GOENV="/home/REDACTED/.config/go/env"
GOEXE=""
GOFLAGS="-buildmode=pie"
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY="REDACTED,*.REDACTED"
GONOSUMDB="REDACTED,*.REDACTED"
GOOS="linux"
GOPATH="/home/REDACTED/Projects/go"
GOPRIVATE="REDACTED,*.REDACTED"
GOPROXY="goproxy-1.REDACTED,direct"
GOROOT="/usr/lib/go-1.14"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.14/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/REDACTED/Projects/github.com/weaveworks/scope/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build194517409=/tmp/go-build -gno-record-gcc-switches"

$ nvm current
v10.24.0

$ node --version
v10.24.0

$ npm --version
6.14.11

$ yarn --version
1.22.5

Logs:

[1]

$ make BUILD_IN_CONTAINER=false
...
cd report && unset GOOS GOARCH; GOGC=off CGO_ENABLED=1 GO111MODULE=off /home/REDACTED/Projects/github.com/weaveworks/scope/vendor/github.com/ugorji/go/codec/codecgen/bin/codecgen_linux_amd64 -d 0 -rt 'netgo unsafe' -u -o report.codecgen.go backcompat.go report.go map_values.go latest_map_generated.go map_keys.go map_helpers.go id_list.go censor.go topology.go string_set.go dns.go controls.go metadata_template.go table.go sets.go marshal.go tools.go metric_template.go id.go node.go datatypes.go node_set.go networks.go metric_row.go metrics.go
codecgen error: error running 'go run codecgen-main-6179.generated.go': exit status 1, console: networks.go:8:2: cannot find package "github.com/k-sone/critbitgo" in any of:
    /usr/lib/go-1.14/src/github.com/k-sone/critbitgo (from $GOROOT)
    /home/REDACTED/Projects/go/src/github.com/k-sone/critbitgo (from $GOPATH)
marshal.go:15:2: cannot find package "github.com/opentracing/opentracing-go" in any of:
    /usr/lib/go-1.14/src/github.com/opentracing/opentracing-go (from $GOROOT)
    /home/REDACTED/Projects/go/src/github.com/opentracing/opentracing-go (from $GOPATH)
marshal.go:16:2: cannot find package "github.com/opentracing/opentracing-go/log" in any of:
    /usr/lib/go-1.14/src/github.com/opentracing/opentracing-go/log (from $GOROOT)
    /home/REDACTED/Projects/go/src/github.com/opentracing/opentracing-go/log (from $GOPATH)
backcompat.go:9:2: cannot find package "github.com/ugorji/go/codec" in any of:
    /usr/lib/go-1.14/src/github.com/ugorji/go/codec (from $GOROOT)
    /home/REDACTED/Projects/go/src/github.com/ugorji/go/codec (from $GOPATH)
node.go:8:2: cannot find package "github.com/weaveworks/common/mtime" in any of:
    /usr/lib/go-1.14/src/github.com/weaveworks/common/mtime (from $GOROOT)
    /home/REDACTED/Projects/go/src/github.com/weaveworks/common/mtime (from $GOPATH)
map_helpers.go:10:2: cannot find package "github.com/weaveworks/ps" in any of:
    /usr/lib/go-1.14/src/github.com/weaveworks/ps (from $GOROOT)
    /home/REDACTED/Projects/go/src/github.com/weaveworks/ps (from $GOPATH)
report.go:9:2: cannot find package "github.com/weaveworks/scope/common/xfer" in any of:
    /usr/lib/go-1.14/src/github.com/weaveworks/scope/common/xfer (from $GOROOT)
    /home/REDACTED/Projects/go/src/github.com/weaveworks/scope/common/xfer (from $GOPATH)
node_set.go:11:2: cannot find package "github.com/weaveworks/scope/test/reflect" in any of:
    /usr/lib/go-1.14/src/github.com/weaveworks/scope/test/reflect (from $GOROOT)
    /home/REDACTED/Projects/go/src/github.com/weaveworks/scope/test/reflect (from $GOPATH)

Makefile:105: recipe for target 'report/report.codecgen.go' failed
make: *** [report/report.codecgen.go] Error 1

[2]

$ grep "github.com/k-sone/critbitgo" go.mod
    github.com/k-sone/critbitgo v1.2.0

$ ls vendor/github.com/k-sone/critbitgo/
CHANGES.md  critbit.go  LICENSE  map.go  net.go  README.md

[3]

$ env GOPROXY=proxy.golang.org go mod vendor
go: github.com/mjibson/appstats@v0.0.0-20151004071057-0542d5f0e87e: reading https://proxy.golang.org/github.com/mjibson/appstats/@v/v0.0.0-20151004071057-0542d5f0e87e.mod: 410 Gone
    server response:
    not found: github.com/mjibson/appstats@v0.0.0-20151004071057-0542d5f0e87e: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /tmp/gopath/pkg/mod/cache/vcs/444aab7ea271b6130ec26c66b91d9ad882ee59ce867f2107d31a0875343c64c2: exit status 128:
        fatal: could not read Username for 'https://github.com': terminal prompts disabled

There is no need to enable terminal prompts, the repo is just gone.

[4]

$ make prog/scope BUILD_IN_CONTAINER=false
rm -f report/report.codecgen.go; unset GOOS GOARCH; env GOGC=off CGO_ENABLED=1 go build -mod vendor -ldflags "-extldflags \"-static\" -X main.version=2f9ccdc8 -s -w" -tags 'netgo unsafe' ./report # workaround for https://github.com/ugorji/go/issues/145
cd report && unset GOOS GOARCH; GOGC=off CGO_ENABLED=1 /home/REDACTED/Projects/github.com/weaveworks/scope/vendor/github.com/ugorji/go/codec/codecgen/bin/codecgen_linux_amd64 -d 0 -rt 'netgo unsafe' -u -o report.codecgen.go backcompat.go report.go map_values.go latest_map_generated.go map_keys.go map_helpers.go id_list.go censor.go topology.go string_set.go dns.go controls.go metadata_template.go table.go sets.go marshal.go tools.go metric_template.go id.go node.go datatypes.go node_set.go networks.go metric_row.go metrics.go
codecgen error: error running 'go run codecgen-main-4973.generated.go': exit status 1, console: go: github.com/mjibson/appstats@v0.0.0-20151004071057-0542d5f0e87e: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /home/REDACTED/Projects/go/pkg/mod/cache/vcs/444aab7ea271b6130ec26c66b91d9ad882ee59ce867f2107d31a0875343c64c2: exit status 128:
    fatal: could not read Username for 'https://github.com': terminal prompts disabled

Makefile:105: recipe for target 'report/report.codecgen.go' failed
make: *** [report/report.codecgen.go] Error 1

[5]

$ grep "github.com/mjibson/appstats" go.mod
    github.com/mjibson/appstats v0.0.0-20151004071057-0542d5f0e87e // indirect
$ ls vendor/github.com/mjibson/appstats
ls: cannot access 'vendor/github.com/mjibson/appstats': No such file or directory
bboreham commented 3 years ago

See work in progress to update the build at #3833

bboreham commented 3 years ago

I believe this is fixed now.