Closed rjschwei closed 8 years ago
Hi @rjschwei,
We intentionally and explicitly tie the version of ecs-init to our latest tested version of Docker. Currently, the latest version of Docker available in the Amazon Linux AMI's yum repositories is 1.6.2. While we believe that ecs-init (and our Agent) should work without changes on newer versions of Docker, we want to verify that it continues to work before we change the spec file.
Does that help answer your question?
Thanks! Sam
On 08/11/2015 12:43 PM, Samuel Karp wrote:
Hi @rjschwei,
We intentionally and explicitly tie the version of ecs-init to our latest tested version of Docker. Currently, the latest version of Docker available in the Amazon Linux AMI's yum repositories is 1.6.2. While we believe that ecs-init (and our Agent) should work without changes on newer versions of Docker, we want to verify that it continues to work before we change the spec file.
Does that help answer your question?
Yes, but leads to a follow on question. Is there a process by which we can test the functionality with a newer version of docker?
Do the tests that are part of the source code cover this or are there additional test that should be run?
Thanks, Robert
Robert Schweikert MAY THE SOURCE BE WITH YOU Public Cloud Architect LINUX rjschwei@suse.com IRC: robjo
I've pushed a set of functional tests to the dev
branch of the Agent (see https://github.com/aws/amazon-ecs-agent/pull/161, thanks @euank for most of the work!), which should do a decent job of testing whether the Agent functions correctly. Note that you will need a fairly permissive role policy on the instance to run these tests (not sure of the complete set offhand, but at least ecs:StartTask
, ecs:DescribeTasks
, ecs:StopTask
, ecs:RegisterTaskDefinition
, ecs:DescribeTaskDefinition
, ecs:DescribeContainerInstances
, and the AmazonEC2ContainerServiceforEC2Role) as well as a cluster named ecs-functional-tests
in order to actually run the tests. You'll also want to either build the Agent locally or re-tag one that you've pulled to amazon/amazon-ecs-agent:make
since that's the image the tests expect to use.
Edit: Fixed some incorrect policy actions.
On 08/11/2015 03:13 PM, Samuel Karp wrote:
I've pushed a set of functional tests to the
dev
branch of the Agent (see https://github.com/aws/amazon-ecs-agent/pull/161, thanks @euank for most of the work!), which should do a decent job of testing whether the Agent functions correctly. Note that you will need a fairly permissive role policy on the instance to run these tests (not sure of the complete set offhand, but at leastecs:StartTask
,ecs:DescribeTask
,ecs:StopTask
,ecs:RegisterTaskDefinition
,ecs:DescribeTaskDefinition
,ecs:DescribeContainerInstance
, and the AmazonEC2ContainerServiceforEC2Role) as well as a cluster namedecs-functional-tests
in order to actually run the tests. You'll also want to either build the Agent locally or re-tag one that you've pulled toamazon/amazon-ecs-agent:make
since that's the image the tests expect to use.
Great, thanks. Will not get to this right away thus I might come back with more questions.
Thanks for the help, Robert
Robert Schweikert MAY THE SOURCE BE WITH YOU Public Cloud Architect LINUX rjschwei@suse.com IRC: robjo
Awesome. I'm going to close this for now, but feel free to either re-open or open a new issue when you have future questions.
I tried to run the tests from the dev branch on our ecs instance with docker 1.7, however I have problems running the tests because of a missing go package requirement. I got the following:
./scripts/test
+ make get-deps
go get github.com/tools/godep
go get golang.org/x/tools/cover
go get golang.org/x/tools/cmd/cover
+ make gotest
GOPATH="/home/ec2-user/amazon-ecs-init/ecs-init/Godeps/_workspace:/home/ec2-user/golang" go test -short -v -cover ./...
ecs-init/ecs-init.go:21:2: cannot find package "github.com/aws/amazon-ecs-init/ecs-init/config" in any of:
/usr/lib64/go/src/pkg/github.com/aws/amazon-ecs-init/ecs-init/config (from $GOROOT)
/home/ec2-user/amazon-ecs-init/ecs-init/Godeps/_workspace/src/github.com/aws/amazon-ecs-init/ecs-init/config (from $GOPATH)
anything I did wrong here ?
Thanks
It looks like you didn't check out the package into your GOPATH
. From the output you've listed, it looks like you've checked it out into /home/ec2-user/amazon-ecs-init
, but your GOPATH
is set to /home/ec2-user/golang
. Can you try checking it out into /home/ec2-user/golang/src/github.com/aws/amazon-ecs-init
instead?
Also, note that my comment above about tests is referring to the ECS Agent repo rather than this repo, and that the tests have been merged into master
of that repo.
ok great that solves it, I checked it out to ~/golang/src/github.com/aws/amazon-ecs-agent of course you are right for testing the agent I should better checkout the agent code :-)
when running the tests I got some error but this is due the environment as you mentioned. I'm looking into this now but might come back with further questions if that's ok for you ?
Thanks for your help
I called
make test-in-docker
and got:
...
# Privileged needed for docker-in-docker so integ tests pass
docker run -v "/home/ec2-user/golang/src/github.com/aws/amazon-ecs-agent:/go/src/github.com/aws/amazon-ecs-agent" --privileged "amazon/amazon-ecs-agent-test:make"
go get github.com/tools/godep
go get golang.org/x/tools/cmd/cover
go get github.com/golang/mock/mockgen
go get golang.org/x/tools/cmd/goimports
cd misc/netkitten; make
make[1]: Entering directory `/go/src/github.com/aws/amazon-ecs-agent/misc/netkitten'
Cannot connect to the Docker daemon. Is 'docker -d' running on this host?
docker runs on this host
6132 ? Ssl 1:20 /usr/bin/docker -d -H fd://
and the testing procedure already communicated with docker, a new registry container was started
docker ps
3e0f76627601 registry "docker-registry" 13 minutes ago Up 13 minutes 127.0.0.1:51670->5000/tcp test-ecs-registry
8717eeb74a6e amazon/amazon-ecs-agent:latest "/agent" 21 hours ago Up 21 hours 127.0.0.1:51678->51678/tcp ecs-agent
An already running ecs-agent should not conflict with the testing, should it ?
Thanks for reporting; it looks like test-in-docker
is actually broken for me as well. I've been using make test
lately for the unit tests and integration tests; that should still be working properly. We'll look into test-in-docker
and see if we can fix it. Note that the functional tests, per my previous comment, should still be run with make run-functional-tests
on an EC2 instance with appropriate permissions.
The make test target on my system is complaining as follows:
...
open $WORK/github.com/docker/libcontainer/system/_obj/_cgo_gotypes.go: No such file or directory
go tool: no such tool "cover"; to install:
go get code.google.com/p/go.tools/cmd/cover
godep: go exit status 3
Makefile:78: recipe for target 'test' failed
However the cover tool is provided and in the path
ls -l ~/golang/bin/cover
-rwxr-xr-x 1 ec2-user users 5399480 Sep 9 08:09 /home/ec2-user/golang/bin/cover
I didn't dive deeper into this because I'm actually most interested in the functional tests. Thus I started 'make run-functional-tests' which is still running. In my ecs-functional-test cluster I can see that a new container instance was registered, thus it seems to do something...
Thanks
Hmm run-functional-tests failed with
=== RUN TestTest
--- PASS: TestTest (0.00 seconds)
=== RUN TestRunManyTasks
panic: test timed out after 20m0s
followed by a stack trace
When I called the tests I saw that a new container was registered in the ecs-functional-test cluster
Sorry still no luck
@schaefi What was the policy attached to your instance when you tried make run-functional-tests
? As I noted above, you'll need a fairly permissive policy to allow the tests to actually run correctly (the functional tests interact directly with the ECS API). You might try watching docker ps
in another terminal while the tests are running; you should see a number of containers created and stopped as part of the functional tests.
With respect to the cover tool, which version of Go are you using? It looks like the version of Go you have installed expects the cover tool to be located at code.google.com/p/go.tools/cmd/cover
, which is no longer accurate (it should now be at golang.org/x/tools/cmd/cover
).
Here is the role policy I have setup according to your information:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecs:CreateCluster",
"ecs:DeregisterContainerInstance",
"ecs:DiscoverPollEndpoint",
"ecs:Poll",
"ecs:RegisterContainerInstance",
"ecs:Submit*",
"ecs:StartTask",
"ecs:DescribeTask",
"ecs:StopTask",
"ecs:RegisterTaskDefinition",
"ecs:DescribeTaskDefinition",
"ecs:DescribeContainerInstance"
],
"Resource": [
"*"
]
}
]
}
The go version installed is
go-1.3.1-2.7.x86_64
Thanks. It looks like my original comments were slightly wrong. You'll want to change ecs:DescribeTask
to ecs:DescribeTasks
, change ecs:DescribeContainerInstance
to ecs:DescribeContainerInstances
, and add ecs:StartTelemetrySession
. Hopefully this should be enough to unblock you. If you still have issues, I can follow up with a more complete list (and get it correctly documented) but you should be able to unblock yourself by using ecs:*
if absolutely necessary.
If I remember correctly, the cover tool changed between Go 1.3 and 1.4. If you don't want to upgrade Go versions, you should still be able to have it function correctly by removing the -cover
argument from the test command in the Makefile.
Hey good news, tests passed now :)
Thanks a ton for your help I would never have found out these role setup
Great! Please let us know if you need any more help.
Hi,
More of a question than an issue. The spec (packaging/amazon-linux-ami/ecs-init.spec) file contains:
Requires: docker >= 1.6.0, docker <= 1.6.2
Is there a reason newer versions of docker will not work. Given that docker has not had any backward incompatible changes it appears that
Requires: docker >= 1.6.0
should be sufficient.
Thoughts?