Add support for docker proxy

[otahi]

Configure docker to use proxies if it is installed.

Docker daemon needs http_proxy environment when behind proxies in a docker config file as follows:

• RHEL
  • /etc/sysconfig/docker
• Ubuntu
  • /etc/default/docker ( < Ubuntu 14.04)
  • /etc/default/docker.io ( >= Ubuntu 14.04 )
• boot2docker
  • [x] /var/lib/boot2docker/profile

I could not find any official documents docker for proxy. See following references. docker issues 1852 Stack Overflow: Cannot download Docker images behind a proxy

[dduportal]

Hi !

Good works, i'm going to test while you roasted me at implementing this.

I'm currently using boot2docker, and it seems that, when you have vagrant-proxyconf which configure an /etc/profile.d/proxy.sh script, we just have to restart docker service, beacuse it'll use profile script.

So Docker support will mostly depends on which Os and which service management is used (init, systemd, etc.)

What do you think around that ?

[otahi]

Thanks for your comment!

I found /usr/local/etc/init.d/docker for boot2docker. It uses /var/lib/boot2docker/profile so we can add configure to that file.

I will add this to my pull request, so I changed this pull request to 'Work In Progress'.

[tmatilai]

Thanks a lot! Unfortunately I haven't had time to go through this yet, but hopefully have on Thursday.

Anyway, the different behavior of guest OSs should be implemented through guest capabilities. This is a design mistake from my part early on. I haven't got into refactoring it on all action classes, but here we could move the logic to capability classes anyway.

[otahi]

@tmatilai Thank you for your advice. If you give me a direction, i will try to move the logic. Can you give me hints or something?

[otahi]

@dduportal I have added docker proxy config on boot2docker. Please check it.

I think there are some kinds of implementations around booting of docker, we need to care all individuals..

[otahi]

I fixed a problem for false values same as #54.

[tmatilai]

@otahi thanks again for all the work. I've been buried with other things, but hope to recover a bit in a week. If this seems to work for people I guess I could merge this then and we can think about refactoring later.

[otahi]

@tmatilai Thank you for your comment. I agree with your idea. On my environment, it seems working OK. Refactoring later is nice to me, because it is hard to find problems without lots of docker users.

[BugRoger]

+1 for this patch.

I tested the plugin and it works nicely with our corporate http proxy. The only issue I found is that the docker service needs to be restarted manually to pick up the new configuration.

[otahi]

@BugRoger Thank you for testing. I understand the issue. I will fix it with restart docker after configuration.

[otahi]

@BugRoger @tmatilai I have added docker daemon restarting. Please check it.

I have checked this change with the following guests, and it works.

• Ubuntu 14.04
• Centos 6.5
• boot2docker 0.8.0

It works even if it is a first boot.

[vagrant@localhost ~]$ sudo strings /proc/`pgrep docker`/environ | grep HTTP_PROXY
HTTP_PROXY=http://192.168.0.2:3128/
[vagrant@localhost ~]$

[johnbellone]

@otahi Does docker use HTTP_PROXY instead of http_proxy? Most UNIX utilities tend to favor the latter.

[dduportal]

I'll take a look behind my company proxy this week (under Cloudstack, and on Windows 7 x64).

[dduportal]

Hi, excuse the ruby's noob question but : how to compile and run the plugin ?

I installed a basic container with ruby 2.0.0/gem from RVM :

FROM debian:wheezy
MAINTAINER damien.duportal@gmail.com

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get install -y wget curl vim procps dnsutils telnet htop

# Install ruby from RVM
RUN curl -sSL https://get.rvm.io | bash -s stable --ruby=2.0.0

RUN apt-get install -y git

RUN /usr/local/bin/gem install bundler

# Configure
ENV PATH /usr/local/bin:$PATH

Where i cloned the two repositories (tmatilai's one and otahi's one) into.

Then i ran a bundle install into the repos, but what to do then ? I'm stuck, with no dev docs to begin with.

[BugRoger]

You need to install the plugin in Vagrant. I assume you don't have your Vagrant inside of a docker container. This would do it on the vagrant host:

git clone https://github.com/otahi/vagrant-proxyconf
cd vagrant-proxyconf
git checkout docker
bundle install
rake build
vagrant plugin install pkg/vagrant-proxyconf-1.3.2.dev.gem

[dduportal]

Well OK, i missed the rake release (didn't know where this tool was user in the build toolchain).

Thanks for the response ! I'll give a shoot these days.

(In fact, i use docker for sharing and reusing the compilation env => From git clone to rake release, all of that will be run into a container, ii's protable and versionnable. Then, i'll copy the generated artefacts to a physical host running vagrant, virtualbox, docker, etc... If i want to be tied to docker only, i can also test the plugin inside my container, with things like https://github.com/jpetazzo/dind, vagrant and only the docker provider :-) )

[BugRoger]

@johnbellone Docker just uses the http transport from Go's standard library. And that one doesn't care about the case of the env variables

http://golang.org/src/pkg/net/http/transport.go#L102 https://github.com/dotcloud/docker/blob/master/registry/registry.go#L50

[tmatilai]

Even though rake release also builds the gem, it's purpose is to release the version to rubygems.org. To only build the gem, execute bundle exec rake build.

But you don't even need to use rake and system ruby, as you can just use Vagrant's embedded ruby environment. The instructions can be found in the wiki.

[dduportal]

@tmatilai thanks a lot, i missed there was a wiki, shame on me !

[dduportal]

FROM debian:wheezy
MAINTAINER damien.duportal@gmail.com

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get install -y wget curl vim procps dnsutils telnet htop

# Install ruby tools
RUN curl -sSL https://get.rvm.io | bash -s stable --ruby=2.0.0
RUN apt-get install -y git 
ENV PATH /usr/local/rvm/rubies/ruby-2.0.0-p481/bin/:$PATH
RUN gem install bundler
RUN mkdir -p /app

# Fetch and compile the project
WORKDIR /app
RUN git clone https://github.com/otahi/vagrant-proxyconf

WORKDIR /app/vagrant-proxyconf
RUN git checkout docker
RUN bundle install
RUN rake build

CMD cat $(find /app/vagrant-proxyconf/pkg -type f -name "*.gem")

Under Mac OS 10.9.3 with boot2docker 1.0.0 =>

docker build -t proxyconf-test .
docker run --rm proxyconf > vagrant-proxyconf.dev.gem
vagrant plugin install vagrant-proxyconf.dev.gem

First test with a local proxy (basic polipo image) works fine.

I'll try under Windows Wednesday.

Anyway, good job !

[otahi]

@johnbellone @BugRoger Thank you for your check and giving a reference. I have confirmed http_proxy has same result.

[vagrant@localhost ~]$ sudo strings /proc/`pgrep docker`/environ | grep http_proxy
http_proxy=http://192.168.0.2:3128/
[vagrant@localhost ~]$

Anyway, I have added unnecessary environments like ftp_proxy and so on. Go library uses only http_proxy and no_proxy, so I think I should delete unnecessary environments. What do you think? > @tmatilai

If it is better to delete, I will modify like this.

        def docker_sed_script
          <<-SED.gsub(/^\s+/, '')
            /^export HTTP_PROXY=/ d
            /^export HTTPS_PROXY=/ d
            /^export FTP_PROXY=/ d
            /^export NO_PROXY=/ d
            /^export http_proxy=/ d
            /^export https_proxy=/ d
            /^export ftp_proxy=/ d
            /^export no_proxy=/ d
          SED
        end

:arrow_down:

        def docker_sed_script
          <<-SED.gsub(/^\s+/, '')
            /^export HTTP_PROXY=/ d
            /^export NO_PROXY=/ d
            /^export http_proxy=/ d
            /^export no_proxy=/ d
          SED
        end

[otahi]

@dduportal Thank you for testing! I want to hear good news from you! :smile:

[otahi]

I have deleted unused environment variables at d8755f4, and it still works on my environment.

[dduportal]

@otahi @tmatilai All working for me : company corporate proxy (dedicated physical appliance), serving only HTTP and HTTPS (FTP thru HTTP supported but no SSH over HTTPS).

I installed the dev gem :

vagrant plugin list
vagrant-cachier (0.7.2)
vagrant-librarian-chef (0.1.5)
vagrant-login (1.0.1, system)
vagrant-proxyconf (1.3.2.dev)
  - Version Constraint: 1.3.2.dev
vagrant-share (1.0.1, system)

Adn then ran my custom boot2docker :

vagrant init dduportal/boot2docker
vagrant up
vagrant ssh
$ docker search nginx => OK
$ docker pull debian:wheezy => OK

My config was a Windows 7 x64, with vagrant 1.6.2

It's all OK for me !

[otahi]

@dduportal Thank you for your time! That's nice!

@tmatilai Can we merge this change?

[tmatilai]

@otahi sure! I'll go through it once more and merge. But can't promise if I manage to do it today.

[tmatilai]

Merged! Thanks to everyone involved!

[otahi]

Thank you @tmatilai and everyone!!

[dduportal]

:+1: