search

lightbend / mesos-spark-integration-tests

Mesos Integration Tests on Docker/Ec2
16 stars 9 forks source link

fix issues for libs loaded for docker in docker (needed by mesos containerizer) #36

Closed skonto closed 8 years ago

skonto commented 8 years ago

Fixes #24 The problem stems from the fact that mesos Docker containerizer needs docker inside docker to run the slaves.

I have tried the following images:

/home/stavros/Downloads/ubuntu-14.04.3-desktop-amd64.iso

/home/stavros/Downloads/ubuntu-15.10-desktop-amd64.iso

15.10:

stavros@vb15:~$ dpkg -L libapparmor1/. /usr /usr/share /usr/share/doc /usr/share/doc/libapparmor1 /usr/share/doc/libapparmor1/copyright /usr/share/doc/libapparmor1/changelog.Debian.gz /lib /lib/x86_64-linux-gnu /lib/x86_64-linux-gnu/libapparmor.so.1.3.0 /lib/x86_64-linux-gnu/libapparmor.so.1 stavros@vb15:~$ dpkg -L libdevmapper1.02.1

/. /lib /lib/x86_64-linux-gnu /lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 /usr /usr/share /usr/share/doc /usr/share/doc/libdevmapper1.02.1 /usr/share/doc/libdevmapper1.02.1/changelog.Debian.gz /usr/share/doc/libdevmapper1.02.1/copyright

stavros@thinkpad:~/workspace/dev/spark-it/mesos-spark-integration-tests$ dpkg -L libapparmor1 /. /usr /usr/share /usr/share/doc /usr/share/doc/libapparmor1 /usr/share/doc/libapparmor1/changelog.Debian.gz /usr/share/doc/libapparmor1/copyright /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/libapparmor.so.1.1.0 /usr/lib/x86_64-linux-gnu/libapparmor.so.1 stavros@thinkpad:~/workspace/dev/spark-it/mesos-spark-integration-tests$ dpkg -L libdevmapper1.02.1 /. /usr /usr/share /usr/share/doc /usr/share/doc/libdevmapper1.02.1 /usr/share/doc/libdevmapper1.02.1/copyright /usr/share/doc/libdevmapper1.02.1/changelog.Debian.gz /lib /lib/x86_64-linux-gnu /lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 stavros@thinkpad:~/workspace/dev/spark-it/mesos-spark-integration-tests$

Although libapparmor has different version in 14.04 LTS and 15.10, only libdevmapper breaks the docker binary when loaded as a volume inside a docker container. The error is (if you try "docker ps" inside the container): docker: /lib/x86_64-linux-gnu/libdevmapper.so.1.02.1: version `DM_1_02_97' not found (required by docker)

In general as a principle it should be best to avoid loading specific libs from host. A lib-independant approach can be found here so docker can be run in docker: https://github.com/jpetazzo/dind/blob/master/Dockerfile https://github.com/jpetazzo/dind/blob/master/wrapdocker This might suit our purposes since the images usues standard ubuntu 14.04. Right now to the extent of my knowledge image docker:dind-1.9.1 (https://hub.docker.com/_/docker/) which officially supports docker in docker natively uses Alpine Linux and does not fit our purposes.

On the other hand, the fix chosen is to detect for linux OSs the version of that library and load it inside it docker. Sometimes you dont need to have the whole setup for docker in docker: https://github.com/kubernetes/kubernetes/pull/17268 https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ The last link is from the guys implemented the official docker in docker.

In the future we might need to consider docker in docker to avoid lib incompatibilities with future Ubuntu releases or other OSs which we might support.

dragos commented 8 years ago

@mgummelt can you comment if this fixes your issues?

mgummelt commented 8 years ago

works! thank you!

dragos commented 8 years ago

Great, I'm in favor of merging this, what do you guys think? /cc @skonto @skyluc @nraychaudhuri

nraychaudhuri commented 8 years ago

+1

skonto commented 8 years ago

should we merge it?

dragos commented 8 years ago

I was under the impression @skyluc wanted to have a look, but I think we can just merge. I'll do it know.