Installer failing when setting up CoG

[nathanlcarlson]

The error causing it to fail is

Installed /usr/local/conda/envs/esgf-pub/lib/python2.7/site-packages/cog-3.11-py2.7.egg
Processing dependencies for cog==3.11
Finished processing dependencies for cog==3.11

*******************************
Setting up Transfer API Client
******************************* 

gcc -o mkproxy mkproxy.c -lcrypto
Traceback (most recent call last):
  File "esg_node.py", line 441, in <module>
    main()
  File "esg_node.py", line 323, in main
    system_component_installation(esg_dist_url, node_type_list)
  File "esg_node.py", line 210, in system_component_installation
    esg_cog.main()
  File "/home/jenkins/repos/esgf-installer/index_node/esg_cog.py", line 169, in main
    setup_cog()
  File "/home/jenkins/repos/esgf-installer/index_node/esg_cog.py", line 131, in setup_cog
    transfer_api_client_python(os.path.join(COG_DIR, "transfer-api-client-python"))
  File "/home/jenkins/repos/esgf-installer/index_node/esg_cog.py", line 75, in transfer_api_client_python
    shutil.copyfile("mkproxy", "/usr/local/conda/envs/esgf-pub/lib/python2.7/site-packages/globusonline/transfer/api_client/x509_proxy/mkproxy")
  File "/usr/local/conda/envs/esgf-pub/lib/python2.7/shutil.py", line 97, in copyfile
    with open(dst, 'wb') as fdst:
IOError: [Errno 2] No such file or directory: '/usr/local/conda/envs/esgf-pub/lib/python2.7/site-packages/globusonline/transfer/api_client/x509_proxy/mkproxy'

But prior to this the installer attempts to do the equivalent of pip install -r requirements.txt, but is failing with errors such as the following:

Collecting django==1.10.4
  Downloading https://files.pythonhosted.org/packages/71/37/581a00bbc4571526ce88ef517c0c02ca7575ac2ae8a3671161d2aa14b740/Django-1.10.4-py2.py3-none-any.whl (6.8MB)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/tmp/pip-req-tracker-HG5yoW/25c5093ede5bd5ac73f7114d1f241f0c397aac863b689092c1f7a9e0'

Collecting django-grappelli==2.8.1
  Downloading https://files.pythonhosted.org/packages/04/2a/3de5f9a1cdc35e52b0306b8e591bb2586af8375c44880e6f903496ff75c2/django-grappelli-2.8.1.tar.gz (2.3MB)
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/tmp/pip-req-tracker-HG5yoW/0542e2f0b94a635429a874af38ef99a9fa2d1b0fe01e2a0a4d7c913d'

Invalid requirement: '#django-openid-auth==0.7', # must be installed independently from fork'
Traceback (most recent call last):
  File "/usr/local/conda/envs/esgf-pub/lib/python2.7/site-packages/pip/_internal/req/req_install.py", line 253, in from_line
    req = Requirement(req)
  File "/usr/local/conda/envs/esgf-pub/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py", line 97, in __init__
    requirement_string[e.loc:e.loc + 8]))
InvalidRequirement: Invalid requirement, parse error at "'#django-'"

[nathanlcarlson]

Regarding the failing pip install -r requirements.txt. I believe this has something to do with using the pip module. The new pip install functions in #514 will likely resolve that.

[nathanlcarlson]

Regarding the second error, it seems that the https://github.com/globusonline/transfer-api-client-python repository is dual purposed. It contains the python package, but it also contains some c code which is being compiled and installed with make install. Note that the repo above has this note in the README

NOTE: This package is no longer being updated, use the Globus SDK for Python instead.

[nathanlcarlson]

I am going to resolve this issue the way we are currently resolving issues, but am going to create a new issue for the fact that Globus wants developers to use Globus SDK for Python

[nathanlcarlson]

514 Resolved the failing pip install described above, but brought to light a number of pip install ed module version conflicts between CoG and ESGF components (esgcet, esgf-security, etc). This will require a new issue.