[python2] `pip` may not be using or honoring CA certs

[fnichol]

Copying original issue comment from @bixu in habitat-sh/habitat#945

---

For example:

    Download error on https://pypi.python.org/simple/numpy/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.
c:590) -- Some packages may not be found!
    Couldn't find index page for 'numpy' (maybe misspelled?)
    Download error on https://pypi.python.org/simple/: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
 -- Some packages may not be found!
    No local packages or download links found for numpy>=1.6
    Traceback (most recent call last):
      File "<string>", line 1, in <module>

      File "/tmp/pip-build-j0adA9/numexpr/setup.py", line 219, in <module>
        setup_package()
      File "/tmp/pip-build-j0adA9/numexpr/setup.py", line 215, in setup_package
        setup(**metadata)
      File "/hab/pkgs/core/python2/2.7.11/20160620182552/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/hab/pkgs/core/python2/2.7.11/20160620182552/lib/python2.7/site-packages/setuptools/dist.py", line 268, in __init_
_
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/hab/pkgs/core/python2/2.7.11/20160620182552/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_b
uild_eggs
        replace_conflicting=True,
      File "/hab/pkgs/core/python2/2.7.11/20160620182552/lib/python2.7/site-packages/pkg_resources/__init__.py", line 836, in
resolve
        dist = best[req.key] = env.best_match(req, ws, installer)
      File "/hab/pkgs/core/python2/2.7.11/20160620182552/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1081, in
 best_match
        return self.obtain(req, installer)
      File "/hab/pkgs/core/python2/2.7.11/20160620182552/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1093, in
 obtain
        return installer(requirement)
      File "/hab/pkgs/core/python2/2.7.11/20160620182552/lib/python2.7/site-packages/setuptools/dist.py", line 380, in fetch_b
uild_egg
        return cmd.easy_install(req)
      File "/hab/pkgs/core/python2/2.7.11/20160620182552/lib/python2.7/site-packages/setuptools/command/easy_install.py", line
 623, in easy_install
        raise DistutilsError(msg)
    distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('numpy>=1.6')

Relevant plan.sh lines (this is a build plan embedded at private_repo/habitat/plan.sh):

pkg_deps=(core/glibc core/gcc core/python2)
pkg_build_deps=(core/gcc core/make core/python2 core/git)
pkg_bin_dirs=(bin)
pkg_lib_dirs=(lib)
pkg_include_dirs=(include)

do_download(){
  return 0
}

do_verify(){
  return 0
}

do_unpack(){
  return 0
}

do_build(){
  pushd /src > /dev/null
  pwd
  pip install --upgrade pip
  pip install --install-option="--prefix=${pkg_prefix}" $(grep numpy requirements.txt)
  pip install --install-option="--prefix=${pkg_prefix}" $(grep numexpr requirements.txt)
  pip install --install-option="--prefix=${pkg_prefix}" $(grep Cython requirements.txt)
  pip install --install-option="--prefix=${pkg_prefix}" -r requirements.txt
  popd > /dev/null
}

do_install(){
  pushd $PLAN_CONTEXT > /dev/null
  pip install -e .
  popd > /dev/null
}

[bdangit]

anytime you use python you need to

export SSL_CERT_FILE=$(hab pkg path core/cacerts)/ssl/cert.pem

or

export SSL_CERT_FILE=$(pkg_path_for core/cacerts)/ssl/cert.pem

Would be nice to have python and/or pip wrapped up to have this set already or there can be a way for packages to have a must have export env vars setting thats always run right after installation of the hab package.

[bixu]

Is there a way for us to pass environment variables from a package we depend on up the chain? This way if <myorigin>/<python_app> depends on core/python we automatically get SSL_CERT_FILE set (I can't think of how to do this, however).

Maybe the right short-term solution is to include an example plan.sh here (right now the example plans don't cover anything Python): https://github.com/habitat-sh/habitat-example-plans

[eeyun]

Yeah the habitat-example-plans repo is not totally useful just yet. It could be good to create one there. As for passing environment variables from upstream packages - There might be ways to do this at runtime (and I think it would work at buildtime too), but currently it would be dependent on the upstream package baking a env.sh that you could source in your downstream package. But I think that could get scary rather quickly.

I'm not opposed to having the option for sure but we might want to explore what kinds of pain that could create.

[georgemarshall]

This appears to be an issue in core/openssl, copying/linking ssl/cert.pem from core/cacerts to core/openssl seems to fix the issue.