Proxy use when running Galaxy

[chouseknecht]

From @TemperingPick on July 20, 2017 12:27

I'm behind an HTTP proxy when trying to run Galaxy so I can't download packages or docker images without a proxy. I've verified that Docker is working fine behind the proxy, and any packages that it fails on I can pull from the local machine. Is there something I need to do to get the containers to be able to use the proxy?

I had to modify the docker template to build the conductor, but running the django container causes an error when trying to download postgresql. Here is the fuck traceback:

The full traceback is: File "/tmp/ansibleF1CRZ/ansible_modlib.zip/ansible/module_utils/urls.py", line 1039, in fetch_url client_key=client_key) File "/tmp/ansibleF1CRZ/ansible_modlib.zip/ansible/module_utils/urls.py", line 948, in open_url r = urllib_request.urlopen(*urlopen_args) File "/_usr/lib64/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/_usr/lib64/python2.7/urllib2.py", line 429, in open req = meth(req) File "/tmp/ansibleF1CRZ/ansible_modlib.zip/ansible/module_utils/urls.py", line 766, in http_request raise ConnectionError('Failed to connect to %s at port %s: %s' % (self.hostname, self.port, to_native(e))) fatal: [django]: FAILED! => { "changed": false, "failed": true, "invocation": { "module_args": { "attributes": null, "backup": null, "checksum": "", "client_cert": null, "client_key": null, "content": null, "delimiter": null, "dest": "/tmp/pgdg-centos95-9.5-3.noarch.rpm", "directory_mode": null, "follow": false, "force": false, "force_basic_auth": false, "group": null, "headers": null, "http_agent": "ansible-httpget", "mode": null, "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "sha256sum": "", "src": null, "timeout": 10, "tmp_dest": null, "unsafe_writes": null, "url": "https://download.postgresql.org/pub/repos/yum/9.5/redhat/rhel-7-x86_64/pgdg-centos95-9.5-3.noarch.rpm", "url_password": null, "url_username": null, "use_proxy": true, "validate_certs": true } }, "msg": "Failed to connect to download.postgresql.org at port 443: [Errno 101] Network is unreachable" } to retry, use: --limit @/tmp/tmpcJ3UCs/playbook.retry

PLAY RECAP ***** django : ok=1 changed=0 unreachable=0 failed=1

2017-07-20T12:19:48.102279 Error applying role! [container.core] caller_file=/_ansible/container/core.py caller_func=apply_role_to_container caller_line=627 engine=<container.docker.engine.Engine object at 0x3689950> exit_code=2 playbook=[{'hosts': u'django', 'roles': ['django-role'], 'vars': {u'galaxy_project_dir': u'/galaxy', u'galaxy_rabbitmq_pass': u'galaxy', u'galaxy_postgres_user': u'galaxy', u'galaxy_postgres_db': u'galaxy', u'galaxy_env': u'DEV', u'galaxy_site_name': u'localhost', u'galaxy_user': u'django', u'galaxy_admin_password': u'admin', u'galaxy_email_username': u'', u'galaxy_venv': u'/venv', u'galaxy_accounts_handler': u'console', u'galaxy_send_email': False, u'galaxy_site_aliases': [u'localhost', u'127.0.0.1', u'0.0.0.0'], u'galaxy_configure_supervisor': False, u'galaxy_email_port': 0, u'galaxy_create_superuser': True, u'galaxy_allauth_handler': u'console', u'galaxy_admin_username': u'admin', u'galaxy_rabbitmq_vhost': u'galaxy', u'galaxy_email_password': u'', u'galaxy_main_handler': u'console', u'galaxy_rabbitmq_user': u'galaxy', u'galaxy_django_handler': u'django_logfile', u'galaxy_postgres_password': u'galaxy', u'galaxy_site_env': u'DEV', u'galaxy_email_hostname': u''}}] 2017-07-20T12:19:48.105360 Playbook run finished. [container.core] caller_file=/_ansible/container/core.py caller_func=conductorcmd_build caller_line=753 exit_code=2 Traceback (most recent call last): File "/usr/bin/conductor", line 11, in load_entry_point('ansible-container', 'console_scripts', 'conductor')() File "/_ansible/container/init.py", line 19, in wrapped return fn(args, kwargs) File "/_ansible/container/cli.py", line 379, in conductor_commandline params) File "/_ansible/container/init.py", line 19, in wrapped return fn(args, **kwargs) File "/_ansible/container/core.py", line 755, in conductorcmd_build raise RuntimeError('Build failed.') RuntimeError: Build failed. 2017-07-20T08:19:48.244033 Conductor terminated. Cleaning up. [container.docker.engine] caller_file=/home/USER/ansible-container-develop/container/docker/engine.py caller_func=await_conductor_command caller_line=356 command_rc=1 conductor_id=u'7f380338f64b7102240314833e44022675e7b8b0f0e24b02f561e2cd93e7a839' save_container=False 2017-07-20T08:19:48.259581 Conductor exited with status 1 [container.cli] caller_file=/home/USER/ansible-container-develop/container/cli.py caller_func=call caller_line=291 make: *** [build] Error 1

Copied from original issue: ansible/galaxy-issues#274

[chouseknecht]

@TemperingPick

Not sure there is a programmatic way to do what you need. The first step is to add the http_proxy and/or https_proxy environment variables to the container.yml file, like so:

version: "2"
settings:
  save_conductor_container: false
  conductor:
    environment:
      http_proxy: http://my-proxy-server
      https_proxy: https://my-proxy_server
  vars_files:
  - develop.yml

services:
  django:
    ... 
    environment: 
      - C_FORCE_ROOT=1
      - http_proxy=http://my-proxy-server
      - https_proxy=https://my-proxy-server 
    ....

  gulp:
    ...
    environment: 
      - http_proxy=http://my-proxy-server
      - https_proxy=https://my-proxy-server 
    ...

registries: {}

[chouseknecht]

From @TemperingPick on July 24, 2017 16:40

@chouseknecht

Is there any where else that I might need to add the environmental variables? I've added them to the main container.yml but the get_url module is still failing even when it says use_proxy=true. I know it's something with the containers because I can wget the file from the server.

[chouseknecht]

@TemperingPick

As an experiment, you might try modifying the task directly, so that it looks like the following:

- name: Get the file
  get_url:
    url: ...
    dest: .... 
  environment:
    http_proxy: http://my-proxy-server

See http://docs.ansible.com/ansible/latest/playbooks_environment.html for more details, if needed.

I was hoping to avoid modifying the tasks by setting the environment variables at the service level in container.yml.

[chouseknecht]

From @TemperingPick on July 24, 2017 19:56

@chouseknecht Doesn't look like get_url supports the environment parameter. It only supports the use_proxy which is set as true.

" "msg": "Unsupported parameters for (get_url) module: environment. Supported parameters include: attributes,backup,checksum,client_cert,client_key,content,delimiter,dest,directory_mode,follow,force,force_basic_auth,group,headers,http_agent,mode,owner,regexp,remote_src,selevel,serole,setype,seuser,sha256sum,src,timeout,tmp_dest,unsafe_writes,url,url_password,url_username,use_proxy,validate_certs""

[chouseknecht]

@TemperingPick

It's not a module parameter. Take a look at the example below. The module name and environment should be at the same indentation level.

- apt: name=cobbler state=installed
  environment:
    http_proxy: http://proxy.example.com:8080

[chouseknecht]

From @TemperingPick on July 25, 2017 2:57

@chouseknecht

Yep that did it! Thanks so much!! If there's a place for it I'd love to document which steps needed to have the environment step added so no one else needs to try and fight with all of this.