search

pyca / cryptography

cryptography is a package designed to expose cryptographic primitives and recipes to Python developers.
https://cryptography.io
Other
6.66k stars 1.53k forks source link

Problem with cryptography on M1 Mac & Python 2.7 #6161

Closed darrennoble closed 3 years ago

darrennoble commented 3 years ago

I'm trying to run fabric on my M1 mac with python 2.7 (it's needed for work until we finish porting to go) and I get cryptography errors with openssl. I installed open ssl via brew (brew install openssl@1.1) although it appears to be using an openssl that shipped with Mac OS and a cryptography that shipped with Mac OS located at /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python

When I run fab I get this runtime error in cryptography:

from cryptography.hazmat.backends import default_backend
Traceback (most recent call last):
  File "/Users/darren/Library/Python/2.7/bin/fab", line 5, in <module>
    from fabric.main import program
  File "/Users/darren/Library/Python/2.7/lib/python/site-packages/fabric/__init__.py", line 3, in <module>
    from .connection import Config, Connection
  File "/Users/darren/Library/Python/2.7/lib/python/site-packages/fabric/connection.py", line 16, in <module>
    from paramiko.agent import AgentRequestHandler
  File "/Users/darren/Library/Python/2.7/lib/python/site-packages/paramiko/__init__.py", line 22, in <module>
    from paramiko.transport import SecurityOptions, Transport
  File "/Users/darren/Library/Python/2.7/lib/python/site-packages/paramiko/transport.py", line 129, in <module>
    class Transport(threading.Thread, ClosingContextManager):
  File "/Users/darren/Library/Python/2.7/lib/python/site-packages/paramiko/transport.py", line 190, in Transport
    if KexCurve25519.is_available():
  File "/Users/darren/Library/Python/2.7/lib/python/site-packages/paramiko/kex_curve25519.py", line 30, in is_available
    X25519PrivateKey.generate()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/cryptography/hazmat/primitives/asymmetric/x25519.py", line 39, in generate
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/cryptography/hazmat/backends/openssl/__init__.py", line 7, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/cryptography/hazmat/backends/openssl/backend.py", line 117, in <module>
    from cryptography.hazmat.bindings.openssl import binding
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: dlopen(/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/cryptography/hazmat/bindings/_openssl.so, 2): Symbol not found: _DTLS_client_method
  Referenced from: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/cryptography/hazmat/bindings/_openssl.so
  Expected in: flat namespace

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/Users/darren/Library/Python/2.7/lib/python/site-packages/paramiko/transport.py", line 120, in _join_lingering_threads
    for thr in _active_threads:
TypeError: 'NoneType' object is not iterable
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/Users/darren/Library/Python/2.7/lib/python/site-packages/paramiko/transport.py", line 120, in _join_lingering_threads
    for thr in _active_threads:
TypeError: 'NoneType' object is not iterable

Is this a problem with my environment or is there a problem with the cryptography that ships with Mac OS's python2 or something else? I tried updating cryptography via pip install -U cryptography but it tells me it's up to date:

pip install -U cryptography
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Requirement already up-to-date: cryptography in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (3.3.2)
Requirement already satisfied, skipping upgrade: six>=1.4.1 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from cryptography) (1.12.0)
Requirement already satisfied, skipping upgrade: cffi>=1.12 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from cryptography) (1.14.3)
Requirement already satisfied, skipping upgrade: enum34 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from cryptography) (1.1.6)
Requirement already satisfied, skipping upgrade: ipaddress in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from cryptography) (1.0.22)
Requirement already satisfied, skipping upgrade: pycparser in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from cffi>=1.12->cryptography) (2.19)

I'm running Python 2.7.16 that came with Mac OS 11.4 and the cryptography that shipped with it.

output of pip freeze:

pip freeze
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
altgraph==0.10.2
appdirs==1.4.4
asn1crypto==0.24.0
bcrypt==3.1.7
bdist-mpkg==0.5.0
bonjour-py==0.3
certifi==2021.5.30
cffi==1.14.3
chardet==4.0.0
configparser==4.0.2
contextlib2==0.6.0.post1
cryptography==3.3.2
distlib==0.3.1
enum34==1.1.6
fabric==2.6.0
filelock==3.0.12
future==0.17.1
idna==2.10
importlib-metadata==2.1.1
importlib-resources==3.3.1
invoke==1.5.0
ipaddress==1.0.22
macholib==1.5.1
matplotlib==1.3.1
modulegraph==0.10.4
numpy==1.8.0rc1
paramiko==2.7.2
pathlib2==2.3.5
powerline-status==2.7
py2app==0.7.3
pycparser==2.19
PyNaCl==1.4.0
pyobjc-core==2.5.1
pyobjc-framework-Accounts==2.5.1
pyobjc-framework-AddressBook==2.5.1
pyobjc-framework-AppleScriptKit==2.5.1
pyobjc-framework-AppleScriptObjC==2.5.1
pyobjc-framework-Automator==2.5.1
pyobjc-framework-CFNetwork==2.5.1
pyobjc-framework-Cocoa==2.5.1
pyobjc-framework-Collaboration==2.5.1
pyobjc-framework-CoreData==2.5.1
pyobjc-framework-CoreLocation==2.5.1
pyobjc-framework-CoreText==2.5.1
pyobjc-framework-DictionaryServices==2.5.1
pyobjc-framework-EventKit==2.5.1
pyobjc-framework-ExceptionHandling==2.5.1
pyobjc-framework-FSEvents==2.5.1
pyobjc-framework-InputMethodKit==2.5.1
pyobjc-framework-InstallerPlugins==2.5.1
pyobjc-framework-InstantMessage==2.5.1
pyobjc-framework-LatentSemanticMapping==2.5.1
pyobjc-framework-LaunchServices==2.5.1
pyobjc-framework-Message==2.5.1
pyobjc-framework-OpenDirectory==2.5.1
pyobjc-framework-PreferencePanes==2.5.1
pyobjc-framework-PubSub==2.5.1
pyobjc-framework-QTKit==2.5.1
pyobjc-framework-Quartz==2.5.1
pyobjc-framework-ScreenSaver==2.5.1
pyobjc-framework-ScriptingBridge==2.5.1
pyobjc-framework-SearchKit==2.5.1
pyobjc-framework-ServiceManagement==2.5.1
pyobjc-framework-Social==2.5.1
pyobjc-framework-SyncServices==2.5.1
pyobjc-framework-SystemConfiguration==2.5.1
pyobjc-framework-WebKit==2.5.1
pyOpenSSL==19.0.0
pyparsing==2.0.1
python-dateutil==1.5
pytz==2013.7
PyYAML==5.4.1
requests==2.25.1
scandir==1.10.0
scipy==0.13.0b1
singledispatch==3.6.2
six==1.12.0
typing==3.10.0.0
urllib3==1.26.6
virtualenv==20.4.7
xattr==0.6.4
zipp==1.2.0

Has anyone else come across this problem or have an ideas on how to fix it?

reaperhulk commented 3 years ago

I can't import the system cryptography on my (non-M1) machine actually, so it may just be that Apple is shipping a semi-broken copy. You should try creating a new virtualenv and installing cryptography in that, but be aware you'll need to compile cryptography and cffi yourself and linker challenges may exist depending upon the way your environment is configured.

darrennoble commented 3 years ago

So I couldn't get fabric to work with Mac OS' built in python2 due to this, but I found out that you can get python2 from the nix package manager (python2 is not available via python.org, brew, or macports for arm/m1). I was then able to install pip and then cryptography. (I had libffi installed from brew and cffi/cryptography was able to use that). Thanks for the help!