building Dockerfile behind proxy / corporate firewall fails

[sfc-gh-eraigosa]

When trying to run the command: docker build --rm -t privatename/docker:registry .

docker fails to run the commands in the Dockerfile. This is mostly because the build is happening behind a proxy/firewall.

I'd like to be able to build the image, not just pull a pre-built image, for testing/experiments and hacking.

Any suggestions? I'm thinking of forking and making some changes to be able to support that.

[dmp42]

Can you copy the output? What commands are failing?

[sfc-gh-eraigosa]

It just hangs forever here:

docker build --rm -t forj/docker:registry .
Sending build context to Docker daemon 730.6 kB
Sending build context to Docker daemon 
Step 0 : FROM ubuntu:14.04
 ---> 8eaa4ff06b53
Step 1 : RUN apt-get update     && apt-get install -y         swig         python-pip         python-dev         libssl-dev         liblzma-dev         libevent1-dev     && rm -rf /var/lib/apt/lists/*
 ---> Running in d87734f4d660

If I add a set -x -v to Step 1 command, I see this output:

docker build --rm -t forj/docker:registry .
Sending build context to Docker daemon 730.6 kB
Sending build context to Docker daemon 
Step 0 : FROM ubuntu:14.04
 ---> 8eaa4ff06b53
Step 1 : RUN set -x -v ; apt-get update     && apt-get install -y         swig         python-pip         python-dev         libssl-dev         liblzma-dev         libevent1-dev     && rm -rf /var/lib/apt/lists/*
 ---> Running in c52924df14b5
+ apt-get update

[sfc-gh-eraigosa]

I'm thinking things like http_proxy, HTTP_PROXY and apt.conf need to be configured to work through proxy.

[dmp42]

Sure. How do you usually handle things? What happens if you try to run apt-get update on a bare ubuntu system from where you are?

[sfc-gh-eraigosa]

Usually we set a proxy file in /etc/profile.d and configure apt.conf with proxy configuration. Let me put a commit together on a copy of my repo to get your thoughts. Shouldn't take long.

[sfc-gh-eraigosa]

@dmp42 this commit on my forked repo makes it much further now, https://github.com/wenlock/docker-registry/commit/81ca0f2b4567ba2cb81ed29895cab7baf8229bc7

Unfortunately, I'm still having a hickup on Step 8, it's complaining about being able to download Werkzeug. Wondering if you've seen this one.

.... <log output removed>....
Step 8 : RUN . /opt/contrib/pip_options.sh     && pip install $PIP_OPTIONS file:///docker-registry#egg=docker-registry[bugsnag,newrelic,cors]
 ---> Running in fde2e9548adb
Unpacking /docker-registry
  Running setup.py (path:/tmp/pip_build_root/docker-registry/setup.py) egg_info for package docker-registry

  Installing extra requirements: 'bugsnag,newrelic,cors'
Requirement already satisfied (use --upgrade to upgrade): docker-registry-core>=2,<3 in /usr/local/lib/python2.7/dist-packages (from docker-registry[bugsnag,newrelic,cors])
Downloading/unpacking backports.lzma==0.0.3,!=0.0.4 (from docker-registry[bugsnag,newrelic,cors])
  Downloading backports.lzma-0.0.3.tar.gz
  Running setup.py (path:/tmp/pip_build_root/backports.lzma/setup.py) egg_info for package backports.lzma
    This is backports.lzma version 0.0.3

Downloading/unpacking blinker==1.3 (from docker-registry[bugsnag,newrelic,cors])
  Running setup.py (path:/tmp/pip_build_root/blinker/setup.py) egg_info for package blinker

Downloading/unpacking Flask==0.10.1 (from docker-registry[bugsnag,newrelic,cors])
  Running setup.py (path:/tmp/pip_build_root/Flask/setup.py) egg_info for package Flask

    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'docs/_themes/.git'
Downloading/unpacking gevent==1.0.1 (from docker-registry[bugsnag,newrelic,cors])
  Running setup.py (path:/tmp/pip_build_root/gevent/setup.py) egg_info for package gevent

Downloading/unpacking gunicorn==19.1 (from docker-registry[bugsnag,newrelic,cors])
Downloading/unpacking PyYAML==3.11 (from docker-registry[bugsnag,newrelic,cors])
  Running setup.py (path:/tmp/pip_build_root/PyYAML/setup.py) egg_info for package PyYAML

Downloading/unpacking requests==2.3.0 (from docker-registry[bugsnag,newrelic,cors])
Downloading/unpacking M2Crypto==0.22.3 (from docker-registry[bugsnag,newrelic,cors])
  Running setup.py (path:/tmp/pip_build_root/M2Crypto/setup.py) egg_info for package M2Crypto

Downloading/unpacking sqlalchemy==0.9.4 (from docker-registry[bugsnag,newrelic,cors])
  Running setup.py (path:/tmp/pip_build_root/sqlalchemy/setup.py) egg_info for package sqlalchemy

    warning: no files found matching '*.jpg' under directory 'doc'
    warning: no files found matching 'distribute_setup.py'
    warning: no files found matching 'sa2to3.py'
    warning: no files found matching 'ez_setup.py'
    no previously-included directories found matching 'doc/build/output'
Requirement already satisfied (use --upgrade to upgrade): setuptools==5.8 in /usr/local/lib/python2.7/dist-packages (from docker-registry[bugsnag,newrelic,cors])
Downloading/unpacking bugsnag>=2.0,<2.1 (from docker-registry[bugsnag,newrelic,cors])
  Downloading bugsnag-2.0.2.tar.gz
  Running setup.py (path:/tmp/pip_build_root/bugsnag/setup.py) egg_info for package bugsnag

Downloading/unpacking Flask-cors>=1.8,<2.0 (from docker-registry[bugsnag,newrelic,cors])
  Downloading Flask_Cors-1.10.2-py2-none-any.whl
Downloading/unpacking newrelic>=2.22,<2.23 (from docker-registry[bugsnag,newrelic,cors])
  Running setup.py (path:/tmp/pip_build_root/newrelic/setup.py) egg_info for package newrelic

Requirement already satisfied (use --upgrade to upgrade): boto==2.34.0 in /usr/local/lib/python2.7/dist-packages (from docker-registry-core>=2,<3->docker-registry[bugsnag,newrelic,cors])
Requirement already satisfied (use --upgrade to upgrade): redis==2.10.3 in /usr/local/lib/python2.7/dist-packages (from docker-registry-core>=2,<3->docker-registry[bugsnag,newrelic,cors])
Requirement already satisfied (use --upgrade to upgrade): simplejson==3.6.2 in /usr/local/lib/python2.7/dist-packages (from docker-registry-core>=2,<3->docker-registry[bugsnag,newrelic,cors])

Downloading/unpacking Werkzeug>=0.7 (from Flask==0.10.1->docker-registry[bugsnag,newrelic,cors])
  Real name of requirement Werkzeug is werkzeug
  Could not find any downloads that satisfy the requirement Werkzeug>=0.7 (from Flask==0.10.1->docker-registry[bugsnag,newrelic,cors])
  Some externally hosted files were ignored (use --allow-external Werkzeug to allow).
Cleaning up...
No distributions at all found for Werkzeug>=0.7 (from Flask==0.10.1->docker-registry[bugsnag,newrelic,cors])
Storing debug log for failure in /root/.pip/pip.log
INFO[0181] The command [/bin/sh -c . /opt/contrib/pip_options.sh     && pip install $PIP_OPTIONS file:///docker-registry#egg=docker-registry[bugsnag,newrelic,cors]] returned a non-zero code: 1 

[sfc-gh-eraigosa]

Also, if i add --alow-external Wekzeug, i get a memory allocation failure

Downloading/unpacking Werkzeug>=0.7 (from Flask==0.10.1->docker-registry[bugsnag,newrelic,cors])
  Real name of requirement Werkzeug is werkzeug
  Running setup.py (path:/tmp/pip_build_root/Werkzeug/setup.py) egg_info for package Werkzeug
    Error [Errno 12] Cannot allocate memory while executing command python setup.py egg_info
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1229, in prepare_files
    req_to_install.run_egg_info()
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 325, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 662, in call_subprocess
    cwd=cwd, env=env)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1223, in _execute_child
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

Storing debug log for failure in /root/.pip/pip.log
INFO[0165] The command [/bin/sh -c . /opt/contrib/pip_options.sh     && pip install $PIP_OPTIONS --allow-external Werkzeug file:///docker-registry#egg=docker-registry[bugsnag,newrelic,cors]] returned a non-zero code: 2 

[dmp42]

Mmmm... pip...

A couple of additional things - this here: https://github.com/docker/docker-registry/issues/821

And this from pip: https://github.com/pypa/pip/issues/1805

That error about werkzeug is weird.

Anything helpful in the pip.log itself?

[sfc-gh-eraigosa]

Ok, I found that my local vm i was using for testing only had 512MB of ram. I increased this to 1024M, and added options --alow-external Wekzeug to my build.

That works now on my side with this commit:

https://github.com/wenlock/docker-registry/commit/5d274b6a3b5ba8d7404561b6f57c8a9085196cae

Any suggestions on both commits before it would be accepted in a pull request?

[dmp42]

@wenlock - your efforts on this are just awesome - thanks a lot for going into this and fixing it. I don't understand why werkzeg install would fail here (doesn't make sense), but that's OT.

For inclusion of this, the bottom line to me is: I would like to reduce as much as possible the added complexity into the dockerfile and would love to see a helper outside instead.

Do you think we can go with something like:

Dockerfile:

ENV http_proxy
ENV https_proxy
ENV HTTP_PROXY
ENV HTTPS_PROXY
ENV ftp_proxy
ENV socks_proxy
ENV no_proxy
ENV PIP_OPTIONS

[...]

pip install $PIP_OPTIONS bla 
pip install $PIP_OPTIONS foo

Then we could just ship into contrib a helper script that would do sthing like below, without cramming it into the dockerfile:

if [ ! -z "$PROXY" ]; then
  export PIP_OPTIONS="--allow-external Werkzeug --proxy $PROXY"
  export HTTP_PROXY=$PROXY
  [...]
fi

and we would instruct people to just ./contrib/setup_proxy.sh foo_proxy then build as usual.

The question that remains is: do we need the apt-conf bits? or can apt-get live happily with the env vars?

What do you think?

[sfc-gh-eraigosa]

Yep, I like that idea to make those ENV's. I do think however the apt-get update simply won't work without an update to the apt.conf. I've not been able to make it happy on my end. I was hoping that maybe someone knew of a plugin kind of like vagrant-proxyconf that could be leveraged here. I ran into a similar issue under vagrant and that seemed to work really smoothly. Wondering how others have solved similar issues with broken proxy's on other dockerfile projects. apt isn't the only config I believe either out there, but it's the one that matters to this particular Dockerfile.

[dmp42]

Let's ping @tianon to find out :-)

[tianon]

Since Dockerfiles can't be parameterized, the cleanest way I've seen this handled is via transparent proxy in iptables (either for all traffic, or just for the mirrors you need explicit proxies for).

[sfc-gh-eraigosa]

@tianon thanks for the input. Yes, I think it would definitely be possible if corporate networking or services were modified. Not always the case for teams though, and certainly not the case for me :<

@dmp42 I was also thinking last night, after looking at the storage drivers, that maybe there might be a way to integrate some pre-run actions in a similar way. Maybe not.... still thinking on this for how we can keep the Dockerfile super trim.

[radenui]

Hello all,

Don't know if this is the right thread for this problem, but as it is definitely related to proxy usage, I submit it here: I run this registry inside a private network that does not allow a direct access to the internet. To make docker server aware of the proxy, I added: export HTTP_PROXY=http://proxy:3128 export HTTPS_PROXY=http://proxy:3128 export NO_PROXY="*.my.domain.int, 10.0.0.0/8" in the /etc/sysconfig/docker file.

It works: I can run the registry directly by using the standard docker run command.

My problem comes later: as the registry is using a backend in S3, I also have to provide the same information into the container itself (and the S3 backend engine in particular) so the registry can actually connect the proxy to access the internet and the S3 bucket location.

Usually, the AWS python SDK is able to interpret same exported variables (HTTP_PROXY, HTTPS_PROXY, NO_PROXY). Would it be possible to enable to container to be passed these variables at start (-e HTTP_PROXY=...) ?

Thanks,

[andersjanmyr]

I also have the same problem, I cannot build my images inside the proxy. I can build if I modify the Dockerfile by adding ENV http_proxy http://my-ip-number:3128/. The problem with this is that then the Dockerfile cannot be used by someone not using a proxy. A solution that I believe could work is if I was able to give environment variables to the build command, similarly to the way I give them to the run command.

# This allows http-client access to my local http proxy.
docker run --env http_proxy=http://10.128.46.150:3128/ my/http-client

# The same could work with build
docker build --env http_proxy=http://10.128.46.150:3128/ -t my/http-client .

What do you think?

[PaulusTM]

I would love a solution as @andersjanmyr suggests.

[b0c1]

@andersjanmyr +1

[byF]

+1

[nwinkler]

+1

[javiervivanco]

+1

[sfc-gh-eraigosa]

question, with the new network plugin features, does it make sense to create a proxy network plugin, or are there plans to fix this in some other way?

[sfc-gh-eraigosa]

i think build --env could still have some usefulness as well but likely maybe a plugin would work better for the proxy use case.

[bjouhier]

@andersjanmyr +1 NTLM proxy is a real PITA and a build --env option would help a lot.

I haven't tried https://github.com/wtsi-hgi/docker-proxify yet but it should also solve the problem.

[nwinkler]

I've tried docker-proxify recently and it was a bit of a pain to use. One of the downsides seems to be that the version of docker bundled in the docker-proxify image is fairly old.

[bjouhier]

I tried it too yesterday and hit the same issue. It needs a refresh.

[jsidhu]

+1

[jonassvatos]

+1

[youngl98]

+1

[danday74]

We are doing ...

ENV http_proxy http://9.9.9.9:9999 ENV https_proxy http://9.9.9.9:9999

and at end of dockerfile ...

ENV http_proxy "" ENV https_proxy ""

This, for now (until docker introduces build env vars), allows the proxy vars to be used for build without publicly exposing them

[lgautier]

A solution might be included in release 1.9.0: https://github.com/docker/docker/issues/14634

[ghostsquad]

:+1:

[friism]

Docker now supports passing --build-env and I believe it's a good approach for HTTP proxies: https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables-build-arg

[zhao-ji]

good job! @friism

[sfc-gh-eraigosa]

Nice job indeed @friism , can we close this issue now :D

[sfc-gh-eraigosa]

cool, i have the power

[softwareklinic]

I have docker 1.12.0 and still unable to communicate to outside world thru corporate proxy - any thoughts?

[ghostsquad]

@softwareklinic are you setting proxy variables and using them at build time?

[softwareklinic]

Tried that i created a new image from.already pulped image by adding/passing.build arguments and i.could see the proxy.variables inside the container bash but still the curl could not connect to outside world.

Sent from my iPhone

On Jul 30, 2016, at 8:55 AM, Wes McNamee notifications@github.com<mailto:notifications@github.com> wrote:

@softwareklinichttps://github.com/softwareklinic are you setting proxy variables and using them at build time?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/docker/docker-registry/issues/890#issuecomment-236372930, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHMCifMJee1BeAFfAzRUFdfhOpCjnlGVks5qa3PZgaJpZM4DPgfo.

[ghostsquad]

Can you build if you hard code the env vars or explicitly use -x parameter?

http://www.cyberciti.biz/faq/linux-unix-curl-command-with-proxy-username-password-http-options/

On Sat, Jul 30, 2016, 11:54 AM Keyur Shah notifications@github.com wrote:

Tried that i created a new image from.already pulped image by adding/passing.build arguments and i.could see the proxy.variables inside the container bash but still the curl could not connect to outside world.

Sent from my iPhone

On Jul 30, 2016, at 8:55 AM, Wes McNamee <notifications@github.com<mailto: notifications@github.com>> wrote:

@softwareklinichttps://github.com/softwareklinic are you setting proxy variables and using them at build time?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://github.com/docker/docker-registry/issues/890#issuecomment-236372930>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AHMCifMJee1BeAFfAzRUFdfhOpCjnlGVks5qa3PZgaJpZM4DPgfo>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/docker/docker-registry/issues/890#issuecomment-236383019, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3JQLiXGyXZfu9bY3N4uZTtZ-7wkkefks5qa53WgaJpZM4DPgfo .

Thanks, Wes

[softwareklinic]

Will try it out today

Sent from my iPhone

On Jul 30, 2016, at 12:23 PM, Wes McNamee notifications@github.com<mailto:notifications@github.com> wrote:

Can you build if you hard code the env vars or explicitly use -x parameter?

http://www.cyberciti.biz/faq/linux-unix-curl-command-with-proxy-username-password-http-options/

On Sat, Jul 30, 2016, 11:54 AM Keyur Shah notifications@github.com<mailto:notifications@github.com> wrote:

Tried that i created a new image from.already pulped image by adding/passing.build arguments and i.could see the proxy.variables inside the container bash but still the curl could not connect to outside world.

Sent from my iPhone

On Jul 30, 2016, at 8:55 AM, Wes McNamee notifications@github.com<mailto:notifications@github.com<mailto: notifications@github.commailto:notifications@github.com>> wrote:

@softwareklinichttps://github.com/softwareklinic are you setting proxy variables and using them at build time?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://github.com/docker/docker-registry/issues/890#issuecomment-236372930>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AHMCifMJee1BeAFfAzRUFdfhOpCjnlGVks5qa3PZgaJpZM4DPgfo>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/docker/docker-registry/issues/890#issuecomment-236383019, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3JQLiXGyXZfu9bY3N4uZTtZ-7wkkefks5qa53WgaJpZM4DPgfo .

Thanks, Wes

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/docker/docker-registry/issues/890#issuecomment-236384908, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AHMCiVOMfU69BYEkyAC__bb7PWQXnx5_ks5qa6SkgaJpZM4DPgfo.

[Jamlee]

it work for me

Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64
 Experimental: false

docker build --build-arg http_proxy=http://my.proxy.url --build-arg foo=bar

FROM busybox
RUN <command that need http_proxy>
ARG --description="foo's description" foo
USER $foo
MARK