botocore explodes when installed alongside pyOpenSSL, ndg-httpsclient and pyasn1

[bhuber]

I'm not an expert with boto or botocore, so apologies in advance if this is a duplicate or whatever, but I can't find anything related in the issues on GH. To reproduce, create a new virtualenv and do the following:

pip install pyOpenSSL==0.14 ndg-httpsclient==0.3.2 pyasn1==0.1.7 botocore==0.38
python
>>> import botocore.vendored.requests

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bhuber/envs/sandbox/local/lib/python2.7/site-packages/botocore/vendored/requests/__init__.py", line 53, in <module>
    from .packages.urllib3.contrib import pyopenssl
  File "/home/bhuber/envs/sandbox/local/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/contrib/pyopenssl.py", line 55, in <module>
    orig_connectionpool_ssl_wrap_socket = connectionpool.ssl_wrap_socket
AttributeError: 'module' object has no attribute 'ssl_wrap_socket'
>>> 

This is a direct manifestation of a bug closed in requests==2.1.0: https://github.com/kennethreitz/requests/issues/1732 . Updating the requests lib would probably fix it.

As a workaround, pip install pyopenssl==0.12 currently fixes it for me.

[mephaust]

I just ran into this bug today, seems to be happening with the following package versions:

botocore==0.57.0
cffi==0.8.6
ndg-httpsclient==0.3.2
pyOpenSSL==0.14
pyasn1==0.1.7

[PiQuer]

+1, downgrading pyopenssl to 0.13.1 (oldest version available in Gentoo) does not help. Python3 doesn't seem to be affected, so for me as awscli user installing the Python3 version of the client is a workaround.

[jdanbrown]

+1 I'm running into this too, trying to get all of the following working together:

• python 2
• awscli
• httpie with SNI support (https://github.com/jakubroztocil/httpie#https)

[jdanbrown]

More info:

awscli breaks with ndg-httpsclient==0.3.2, but http SNI works:

$ python --version
Python 2.7.7

$ pip freeze | ack -i 'awscli|botocore|pyopenssl|pyasn1|ndg-httpsclient'
awscli==1.4.2
botocore==0.62.0
ndg-httpsclient==0.3.2
pyOpenSSL==0.14
pyasn1==0.1.7

$ aws --version
Traceback (most recent call last):
  File "/usr/local/bin/aws", line 15, in <module>
    import awscli.clidriver
  File "/usr/local/lib/python2.7/site-packages/awscli/clidriver.py", line 16, in <module>
    import botocore.session
  File "/usr/local/lib/python2.7/site-packages/botocore/session.py", line 27, in <module>
    import botocore.credentials
  File "/usr/local/lib/python2.7/site-packages/botocore/credentials.py", line 27, in <module>
    from botocore.utils import InstanceMetadataFetcher, parse_key_val_file
  File "/usr/local/lib/python2.7/site-packages/botocore/utils.py", line 19, in <module>
    from botocore.vendored import requests
  File "/usr/local/lib/python2.7/site-packages/botocore/vendored/requests/__init__.py", line 53, in <module>
    from .packages.urllib3.contrib import pyopenssl
  File "/usr/local/lib/python2.7/site-packages/botocore/vendored/requests/packages/urllib3/contrib/pyopenssl.py", line 55, in <module>
    orig_connectionpool_ssl_wrap_socket = connectionpool.ssl_wrap_socket
AttributeError: 'module' object has no attribute 'ssl_wrap_socket'

$ http head https://sni.velox.ch
HTTP/1.1 200 OK
Cache-Control: max-age=0
Content-Type: text/html; charset=utf-8
Date: Wed, 20 Aug 2014 02:32:40 GMT
Expires: Wed, 20 Aug 2014 02:32:40 GMT
Server: Apache/2.4

awscli works with ndg-httpsclient==0.3.1, but http SNI breaks:

$ python --version
Python 2.7.7

$ pip freeze | ack -i 'awscli|botocore|pyopenssl|pyasn1|ndg-httpsclient'
awscli==1.4.2
botocore==0.62.0
ndg-httpsclient==0.3.1
pyOpenSSL==0.14
pyasn1==0.1.7

$ aws --version
aws-cli/1.4.2 Python/2.7.7 Darwin/13.3.0

$ http head https://sni.velox.ch
http: error: SSLError: hostname 'sni.velox.ch' doesn't match either of 'alice.sni.velox.ch', 'carol.sni.velox.ch'

[jamesls]

It sounds like from our side upgrading the version of the requests library we use should fix this issue.

Unfortunately, upgrading to requests 2.3.0 introduced multiple failing tests so it'll take some time for me to go through these test failures and understand where these regressions are coming from.

[phunehehe]

Downgrading to pyopenssl==0.12 didn't help for me. Installing in Python 3 did.

[felixbuenemann]

I had the same issue with awscli, fixed it by replacing the vendored requests lib in botocore with a symlink to a current version in site-packages.

[guss77]

I have the same problem with installing awscli on Ubuntu 15.04. I worked around by doing:

sudo apt-get purge python-ndg-httpsclient
sudo pip install ndg-httpsclient==0.3.1

[Rowanto]

Thanks @guss77 . I am using Debian testing, and those commands also fixed my eb cli. :+1:

[jdanbrown]

:boom: Thanks @jamesls!

• Works for me with awscli==1.7.7 + botocore==0.88.0 + requests==2.5.1
• Works with either ndg-httpsclient==0.3.3 or 0.3.2, but not with 0.3.1 like in my comment above

$ python --version
Python 2.7.7

$ pip freeze | ack -i '^(awscli|botocore|pyopenssl|pyasn1|ndg-httpsclient|requests)\='
awscli==1.7.7
botocore==0.88.0
ndg-httpsclient==0.3.3
pyOpenSSL==0.14
pyasn1==0.1.7
requests==2.5.1

$ aws --version
aws-cli/1.7.7 Python/2.7.7 Darwin/13.4.0

$ http head https://sni.velox.ch
HTTP/1.1 200 OK
Cache-Control: max-age=0
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
Date: Fri, 13 Feb 2015 06:49:51 GMT
Expires: Fri, 13 Feb 2015 06:49:51 GMT
Keep-Alive: timeout=15, max=100
Server: Apache/2.4
Strict-Transport-Security: max-age=10886400; includeSubDomains

[jarrettj]

@guss77 worked on ubuntu 15.04, Thanks.

[memosstilvi]

@guss77 worked on ubuntu 14.04, Thanks.