weaveworks / weave

Simple, resilient multi-host containers networking and more.
https://www.weave.works
Apache License 2.0
6.62k stars 668 forks source link

Weave does not work with RHEL7/CentOS7 docker-1.6 #662

Closed psfales closed 9 years ago

psfales commented 9 years ago

I'm hoping you can help me debug why weave is not working with the stock version of docker-1.6

With the previous docker-1.5, there was a known issue with RHEL7/CentOS7 where docker would not properly mount a socket as a volume. (When we use "-v /var/run/docker.sock:/var/run/docker.sock") the file would not appear as a socket inside the container.

To work around that, I had installed a private copy of the docker-1.6 release candidate. I was hoping that the new 1.6 version would fix the problems. In fact, it does seem to fix that particular issue. But weave still does not work. I can start the weave container ok, but any "docker run" command reports: time="2015-05-14T17:06:32-04:00" level=fatal msg="Error response from daemon: : exit status 2"

If I go back to my private copy of the non-redhat docker-1.6-rc5 then it works!

dpw commented 9 years ago

Hi,

I'm trying to reproduce your problem on centos 7 with docker 1.6 with some simple uses of weave, but I haven't succeeded so far. Could you provide the commands you use to get that "Error response from daemon"?

Thanks.

psfales commented 9 years ago

It's actually quite simple to reproduce. When I sent my earlier mail, I was running the powerstrip plugin which must have hidden the error.

I see noow that I can reproduce it with something as simple as

# weave launch
time="2015-05-14T17:55:41-04:00" level=fatal msg="Error response from daemon: : exit status 2" 

I tried this on 4 different machines, all running CentOS7 and all recently upgraded to the recent docker-1.6.0-11.0.1.el7.centos.x86_64

I thought it might be an out-of-date weave, beacause "weave setup" pulled some new stuff, but it still fails the same way.

But I can do this!

# service docker stop
Redirecting to /bin/systemctl stop  docker.service
# cp /usr/bin/docker-1.6-rc6  /usr/bin/docker
cp: overwrite `usr/bin/docker'? y
# service docker start
Redirecting to /bin/systemctl start  docker.service
# weave launch
b06f12b1a9d0b7154e6f4b09ad11dfc4b4d5eb6ca4d3d87d5bed14152a22f356
# 

Peter Fales Peter "at" fales-lorenz.net

On Thu, May 14, 2015 at 02:43:46PM -0700, David Wragg wrote:

Hi,

I'm trying to reproduce your problem on centos 7 with docker 1.6 with some simple uses of weave, but I haven't succeeded so far. Could you provide the commands you use to get that "Error response from daemon"?

Thanks.

-- Reply to this email directly or [1]view it on GitHub.

References

  1. https://github.com/weaveworks/weave/issues/662#issuecomment-102178511
dpw commented 9 years ago

I'm still not able to reproduce that error: With weave 0.10.0 on centos7, weave launch is successful for me. Which version of weave are you using?

Most likely the problem is related to selinux. I assume that /etc/selinux/config shows that you have selinux set to enforcing and targeted? If you set selinux to permissive with setenforce 0 does the problem go away?

It might be worth doing fixfiles -R docker verify to check that all the files in your docker install have the correct selinux attributes (though don't worry if it reports files under /var/lib/docker/containers/).

I would guess that the RH-built docker is patched with enhanced selinux support, whereas your build was not, accounting for the difference in behaviour.

psfales commented 9 years ago

OK, Thanks and sorry for the trouble. Looks like the issue is my old version of weave.

I was under the mistaken impression that "weave setup" would upgrade me to the newest version of weave. I had tried running "weave setup" and it pulled new docker images, but I didn't get the new version of weave until I pulled the update from https://github.com/weaveworks/weave/releases/download/latest_release/weave

Now it's working for me with the CentOS7 version of docker-1.6!

(For the record, I have not been using selinux on any of our systems, so I don't think that's a factor)

Peter Fales Peter "at" fales-lorenz.net