search

aws / amazon-ecs-agent

Amazon Elastic Container Service Agent
http://aws.amazon.com/ecs/
Apache License 2.0
2.06k stars 602 forks source link

"unable to detect version control system" for mockgen #117

Closed WheresWardy closed 8 years ago

WheresWardy commented 9 years ago

I'm attempting to build the ECS agent on an EC2 instance running Ubuntu 14.04, and getting the following error when it attempts to pull the mockgen dependency, with the error

package code.google.com/p/gomock/mockgen: unable to detect version control system for code.google.com/ path

(I get the same error when trying to pull the dependency locally with Go).

http://code.google.com/p/gomock/mockgen redirects to GitHub now, so possible the dependency location needs updating in this repo?

root@amazon-ecs-agent-1.2.0# make
docker build -t "amazon/amazon-ecs-agent-cert-source:make" misc/certs/
Sending build context to Docker daemon 3.072 kB
Sending build context to Docker daemon
Step 0 : FROM debian:latest
 ---> bf84c1d84a8f
Step 1 : RUN apt-get update &&      apt-get install -y ca-certificates &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 91350f438685
Successfully built 91350f438685
docker run "amazon/amazon-ecs-agent-cert-source:make" cat /etc/ssl/certs/ca-certificates.crt > misc/certs/ca-certificates.crt
Sending build context to Docker daemon 5.024 MB
Sending build context to Docker daemon
Step 0 : FROM golang:1.4
 ---> 1a22368b487a
Step 1 : MAINTAINER Amazon Web Services, Inc.
 ---> Using cache
 ---> 95525eda1a2d
Step 2 : RUN mkdir /out
 ---> Using cache
 ---> 9c4bf2b29597
Step 3 : VOLUME ['/out']
 ---> Using cache
 ---> 0d5232a84ee9
Step 4 : RUN mkdir -p /go/src/github.com/aws/
 ---> Using cache
 ---> 8dfd53fcff05
Step 5 : COPY /scripts/build /scripts/build
 ---> Using cache
 ---> ebd1386ec32c
Step 6 : WORKDIR /go/src/github.com/aws/amazon-ecs-agent
 ---> Using cache
 ---> d2af5fbe8acf
Step 7 : ENTRYPOINT /scripts/build
 ---> Using cache
 ---> cd7444209b3e
Successfully built cd7444209b3e
go get github.com/tools/godep
go get golang.org/x/tools/cmd/cover
go get code.google.com/p/gomock/mockgen
package code.google.com/p/gomock/mockgen: unable to detect version control system for code.google.com/ path
Makefile:95: recipe for target 'get-deps' failed
make: *** [get-deps] Error 1
cat agent/gogenerate/inflections.csv >> agent/Godeps/_workspace/src/github.com/awslabs/aws-sdk-go/internal/model/api/inflections.csv
acs/update_handler/os/mock/filesystem.go:1:1: expected 'package', found 'EOF'
api/mocks/api_mocks.go:1:1: expected 'package', found 'EOF'
ec2/mocks/ec2_mocks.go:1:1: expected 'package', found 'EOF'
engine/dockerclient/mocks/dockerclient_mocks.go:1:1: expected 'package', found 'EOF'
engine/mocks/engine_mocks.go:1:1: expected 'package', found 'EOF'
httpclient/mock/httpclient.go:1:1: expected 'package', found 'EOF'
stats/mock/engine.go:1:1: expected 'package', found 'EOF'
stats/resolver/mock/resolver.go:1:1: expected 'package', found 'EOF'
wsclient/mock/client.go:1:1: expected 'package', found 'EOF'
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
acs/update_handler/os/filesystem.go:18: running "mockgen.sh": exit status 127
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
api/generate_mocks.go:16: running "mockgen.sh": exit status 127
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
ec2/generate_mocks.go:16: running "mockgen.sh": exit status 127
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
engine/generate_mocks.go:16: running "mockgen.sh": exit status 127
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
engine/dockerclient/generate_mocks.go:16: running "mockgen.sh": exit status 127
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
httpclient/httpclient.go:33: running "mockgen.sh": exit status 127
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
stats/engine.go:16: running "mockgen.sh": exit status 127
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
stats/resolver/resolver.go:18: running "mockgen.sh": exit status 127
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: mockgen: command not found
/go/src/github.com/aws/amazon-ecs-agent/scripts/generate/mockgen.sh: line 43: goimports: command not found
wsclient/client.go:92: running "mockgen.sh": exit status 127
godep: go exit status 1
Makefile:59: recipe for target 'gogenerate' failed
make: *** [gogenerate] Error 1
./
./tmp/
8f2da306715d3021818b12b7058b56c6aab77aabc4cea0efddf6bbb69889eb01
Sending build context to Docker daemon 5.024 MB
Sending build context to Docker daemon
Step 0 : FROM amazon/amazon-ecs-scratch:make
 ---> 8f2da306715d
Step 1 : COPY out/amazon-ecs-agent /agent
INFO[0000] out/amazon-ecs-agent: no such file or directory
make: *** [docker] Error 1
WheresWardy commented 9 years ago

Also, even if I correct the dependencies, I still get issues:

root@amazon-ecs-agent-1.2.0# make
docker build -t "amazon/amazon-ecs-agent-cert-source:make" misc/certs/
Sending build context to Docker daemon 3.072 kB
Sending build context to Docker daemon
Step 0 : FROM debian:latest
 ---> bf84c1d84a8f
Step 1 : RUN apt-get update &&      apt-get install -y ca-certificates &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 91350f438685
Successfully built 91350f438685
docker run "amazon/amazon-ecs-agent-cert-source:make" cat /etc/ssl/certs/ca-certificates.crt > misc/certs/ca-certificates.crt
Sending build context to Docker daemon 5.024 MB
Sending build context to Docker daemon
Step 0 : FROM golang:1.4
 ---> 1a22368b487a
Step 1 : MAINTAINER Amazon Web Services, Inc.
 ---> Using cache
 ---> 95525eda1a2d
Step 2 : RUN mkdir /out
 ---> Using cache
 ---> 9c4bf2b29597
Step 3 : VOLUME ['/out']
 ---> Using cache
 ---> 0d5232a84ee9
Step 4 : RUN mkdir -p /go/src/github.com/aws/
 ---> Using cache
 ---> 8dfd53fcff05
Step 5 : COPY /scripts/build /scripts/build
 ---> Using cache
 ---> ebd1386ec32c
Step 6 : WORKDIR /go/src/github.com/aws/amazon-ecs-agent
 ---> Using cache
 ---> d2af5fbe8acf
Step 7 : ENTRYPOINT /scripts/build
 ---> Using cache
 ---> cd7444209b3e
Successfully built cd7444209b3e
go get github.com/tools/godep
go get golang.org/x/tools/cmd/cover
go get github.com/golang/mock/gomock
go get github.com/golang/mock/mockgen
go get golang.org/x/tools/cmd/goimports
cat agent/gogenerate/inflections.csv >> agent/Godeps/_workspace/src/github.com/awslabs/aws-sdk-go/internal/model/api/inflections.csv
acs/update_handler/os/mock/filesystem.go:1:1: expected 'package', found 'EOF'
api/mocks/api_mocks.go:1:1: expected 'package', found 'EOF'
ec2/mocks/ec2_mocks.go:1:1: expected 'package', found 'EOF'
engine/dockerclient/mocks/dockerclient_mocks.go:1:1: expected 'package', found 'EOF'
engine/mocks/engine_mocks.go:1:1: expected 'package', found 'EOF'
httpclient/mock/httpclient.go:1:1: expected 'package', found 'EOF'
stats/mock/engine.go:1:1: expected 'package', found 'EOF'
stats/resolver/mock/resolver.go:1:1: expected 'package', found 'EOF'
wsclient/mock/client.go:1:1: expected 'package', found 'EOF'
git checkout -- agent/Godeps/_workspace/src/github.com/awslabs/aws-sdk-go/internal/model/api/inflections.csv
fatal: Not a git repository (or any parent up to mount point /go/src/github.com/aws/amazon-ecs-agent)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Makefile:59: recipe for target 'gogenerate' failed
make: *** [gogenerate] Error 128
./
./tmp/
9b8ed2e6ee0db6bd1f11d6fa08fe8460d61f20bf819689510eb8576d4898f23a
Sending build context to Docker daemon 5.062 MB
Sending build context to Docker daemon
Step 0 : FROM amazon/amazon-ecs-scratch:make
 ---> 9b8ed2e6ee0d
Step 1 : COPY out/amazon-ecs-agent /agent
INFO[0000] out/amazon-ecs-agent: no such file or directory
make: *** [docker] Error 1
euank commented 9 years ago

The issue with gomock's import location has been fixed in the dev branch now (see #115).

The expected 'package', found 'EOF' errors are not actually necessarily fatal. In this case the previous mockgen run probably left you with empty files and it's replacing them on this run; running again might no longer show those errors.

The actual error, of fatal: Not a git repository, is what's failing the build. Mind trying again with the .git directory present?

Thanks for the information. I've been meaning to do some rework of how we build, and this serves as a good reminder of why :smiley:. If you run into more rough edges I'd like to hear about them as well. Best, Euan

WheresWardy commented 9 years ago

Thanks for the info. The reason the .git directory wasn't present is because we pull the release tarball from the Releases page, which won't have the git checkout. Are you saying we now need to clone the repo instead for building in future?

euank commented 9 years ago

Sorry for the slow response here. The build system does assume you're building in the git repository. It does this for two reasons:

  1. To embed the commit hash as part of the version information
  2. To 'reset' changes only needed during the build process (e.g. inflections.csv changes, version changes)

Both of those could be changed to not require git. Does doing a clone and checkout of the version tag for that release sound like a good solution to you, or would you rather have us also make our build work outside of git as well?

WheresWardy commented 9 years ago

A clone and checkout shouldn't be a problem, I can update our build process accordingly. The release tarball would certainly be easier in the long run if that's possible, as it means we can mirror the tarball and build it environments where we don't have git for whatever reason (like making PCI environments easier to manage etc.)

euank commented 8 years ago

The build system has since been refactored a good bit.

Several build targets (build-in-docker, static, gobuild) with neither the .git folder nor networking (e.g. docker run -it -v $(pwd):/go/src/github.com/aws/amazon-ecs-agent --workdir=/go/src/github.com/aws/amazon-ecs-agent --net=none golang:1.4 works for me for the v1.6.0 release tarball).

There's still a git dependency for make release and git is necessary to get a proper build version (since it includes the git short hash), but it's a lot better now.

I'm closing this issue since I think the refactor of the build system should resolve the problems you originally had.

Best, Euan