nchammas / flintrock

A command-line tool for launching Apache Spark clusters.
Apache License 2.0
636 stars 116 forks source link

Get AttributeError: type object 'EllipticCurvePublicKey' has no attribute 'from_encoded_point' #305

Closed banshee closed 4 years ago

banshee commented 4 years ago

Launch a cluster, say yes to Do you want to terminate the 3 instances created by this operation? [Y/n]: y

Get this error:

james@gig:~/workspace/awsemrlocaldocker$ flintrock launch teststuff Warning: Downloading Spark from an Apache mirror. Apache mirrors are often slow and unreliable, and typically only serve the most recent releases. We strongly recommend you specify a custom download source. For more background on this issue, please see: https://github.com/nchammas/flintrock/issues/238 Requesting 3 spot instances at a max price of $0.007... 0 of 3 instances granted. Waiting... All 3 instances granted. Do you want to terminate the 3 instances created by this operation? [Y/n]: y Terminating instances... Traceback (most recent call last): File "/home/james/.local/bin/flintrock", line 10, in sys.exit(main()) File "/home/james/.local/lib/python3.7/site-packages/flintrock/flintrock.py", line 1187, in main cli(obj={}) File "/home/james/.local/lib/python3.7/site-packages/click/core.py", line 764, in call return self.main(args, kwargs) File "/home/james/.local/lib/python3.7/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/home/james/.local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/james/.local/lib/python3.7/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/james/.local/lib/python3.7/site-packages/click/core.py", line 555, in invoke return callback(args, kwargs) File "/home/james/.local/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func return f(get_current_context(), *args, *kwargs) File "/home/james/.local/lib/python3.7/site-packages/flintrock/flintrock.py", line 456, in launch tags=ec2_tags) File "/home/james/.local/lib/python3.7/site-packages/flintrock/ec2.py", line 53, in wrapper res = func(args, kwargs) File "/home/james/.local/lib/python3.7/site-packages/flintrock/ec2.py", line 955, in launch identity_file=identity_file) File "/home/james/.local/lib/python3.7/site-packages/flintrock/core.py", line 625, in provision_cluster run_against_hosts(partial_func=partial_func, hosts=hosts) File "/home/james/.local/lib/python3.7/site-packages/flintrock/core.py", line 492, in run_against_hosts future.result() File "/usr/lib/python3.7/concurrent/futures/_base.py", line 432, in result return self.get_result() File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in get_result raise self._exception File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/home/james/.local/lib/python3.7/site-packages/flintrock/core.py", line 675, in provision_node wait=True) File "/home/james/.local/lib/python3.7/site-packages/flintrock/ssh.py", line 66, in get_ssh_client client.load_system_host_keys() File "/home/james/.local/lib/python3.7/site-packages/paramiko/client.py", line 104, in load_system_host_keys self._system_host_keys.load(filename) File "/home/james/.local/lib/python3.7/site-packages/paramiko/hostkeys.py", line 101, in load e = HostKeyEntry.from_line(line, lineno) File "/home/james/.local/lib/python3.7/site-packages/paramiko/hostkeys.py", line 364, in from_line key = ECDSAKey(data=decodebytes(key), validate_point=False) File "/home/james/.local/lib/python3.7/site-packages/paramiko/ecdsakey.py", line 163, in init key = ec.EllipticCurvePublicKey.from_encoded_point( AttributeError: type object 'EllipticCurvePublicKey' has no attribute 'from_encoded_point'

banshee commented 4 years ago

This seems to have fixed it (found on https://stackoverflow.com/questions/54861026/how-to-silence-ellipticcurvepublicnumbers-encode-point-cryptographydeprecationwa?noredirect=1&lq=1)

(I have no idea if this is a good idea or not; I know almost nothing about the python ecosystem, other than it's mostly a giant hairball of pain and suffering.)

james@gig:~/workspace/awsemrlocaldocker$ pip3 install --upgrade paramiko Requirement already up-to-date: paramiko in /home/james/.local/lib/python3.7/site-packages (2.7.1) Requirement already satisfied, skipping upgrade: pynacl>=1.0.1 in /usr/lib/python3/dist-packages (from paramiko) (1.3.0) Requirement already satisfied, skipping upgrade: bcrypt>=3.1.3 in /home/james/.local/lib/python3.7/site-packages (from paramiko) (3.1.7) Collecting cryptography>=2.5 (from paramiko) Downloading https://files.pythonhosted.org/packages/dc/5e/e1e6991a841a9e88eb848634026199fa92665ae792ee6362a0cfa0ece3dc/cryptography-2.9-cp35-abi3-manylinux1_x86_64.whl (2.7MB) 100% |████████████████████████████████| 2.7MB 742kB/s Requirement already satisfied, skipping upgrade: cffi>=1.1 in /home/james/.local/lib/python3.7/site-packages (from bcrypt>=3.1.3->paramiko) (1.14.0) Requirement already satisfied, skipping upgrade: six>=1.4.1 in /usr/lib/python3/dist-packages (from bcrypt>=3.1.3->paramiko) (1.12.0) Requirement already satisfied, skipping upgrade: pycparser in /home/james/.local/lib/python3.7/site-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko) (2.20) Installing collected packages: cryptography Successfully installed cryptography-2.9

banshee commented 4 years ago

Leaving the bug open because it seems like the install of flintrock should have picked this up as a problem. I'd understand if you want to close this as not-a-flintrock-bug though.

banshee commented 4 years ago

Install of flintrock was by:

pip3 install flintrock

nchammas commented 4 years ago

Hmm, can you reproduce the error and then share the output of pip3 list?

Can you also share the output you got when you first installed Flintrock? I'm not clear on how you got an outdated version of Cryptography with the install.

For the record, if working with Python is painful for you, you can always install the stand-alone version of Flintrock that bundles its own Python interpreter.

banshee commented 4 years ago

After I updated paramiko, the error went away, so unfortunately I can't give you a repro any more. Here's the pip3 list output output though, if it's useful.

Package Version


apturl 0.5.2
asn1crypto 0.24.0
bcrypt 3.1.7
blinker 1.4
boto3 1.10.45
botocore 1.13.45
Brlapi 0.6.7
cached-property 1.5.1
certifi 2018.8.24
cffi 1.14.0
chardet 3.0.4
Click 7.0
command-not-found 0.3
cryptography 2.9
cupshelpers 1.0
decorator 4.3.0
defer 1.0.6
distro 1.3.0
distro-info 0.21ubuntu2
docker 3.4.1
docker-pycreds 0.3.0
dockerpty 0.4.1
docopt 0.6.2
docutils 0.15.2
entrypoints 0.3
Flintrock 1.0.0
httplib2 0.11.3
idna 2.6
ifaddr 0.1.6
jmespath 0.9.5
jsonschema 2.6.0
keyring 17.1.1
keyrings.alt 3.1.1
language-selector 0.1
launchpadlib 1.10.6
lazr.restfulclient 0.14.2
lazr.uri 1.0.3
louis 3.8.0
macaroonbakery 1.2.1
Mako 1.0.7
MarkupSafe 1.1.0
netifaces 0.10.4
numpy 1.16.2
numpy-stl 2.8.0
oauthlib 2.1.0
olefile 0.46
paramiko 2.7.1
pexpect 4.6.0
Pillow 5.4.1
pip 18.1
protobuf 3.6.1
pycairo 1.16.2
pycparser 2.20
pycrypto 2.6.1
pycups 1.9.73
PyGObject 3.32.0
PyJWT 1.7.0
pymacaroons 0.13.0
PyNaCl 1.3.0
PyQt5 5.12.1
pyRFC3339 1.1
pyserial 3.4
python-apt 1.8.5-ubuntu0.3 python-dateutil 2.7.3
python-debian 0.1.34
python-utils 2.2.0
pytz 2018.9
pyxdg 0.25
PyYAML 5.2
reportlab 3.5.18
requests 2.21.0
requests-unixsocket 0.1.5
s3transfer 0.2.1
scipy 1.2.1
screen-resolution-extra 0.0.0
SecretStorage 2.3.1
setuptools 40.8.0
simplejson 3.16.0
six 1.12.0
ssh-import-id 5.7
system-service 0.3
systemd-python 234
texttable 1.6.0
ubuntu-advantage-tools 19.2
ubuntu-drivers-common 0.0.0
ufw 0.36
unattended-upgrades 0.1
urllib3 1.24.1
usb-creator 0.3.3
wadllib 1.3.3
websocket-client 0.53.0
wheel 0.32.3
xkit 0.0.0
zeroconf 0.21.3

banshee commented 4 years ago

And I don't remember seeing errors or warnings when I did the initial pip3 install flintrock. If there was output, though, I didn't capture it in any way.

banshee commented 4 years ago

The only other information I can think of is that this is an initial install of flintrock, not an update. I never used previous versions.

nchammas commented 4 years ago

Hmm, it will be difficult to figure out what happened if you can't reproduce the issue. What I can say is that Flintrock already specifies Paramiko 2.7, so you should not have needed to do a separate upgrade of that package. The correct version of Paramiko would have been installed alongside Flintrock.

So I suspect something was off either during the initial installation of Flintrock, or perhaps at some point after, causing the wrong version of Paramiko to get installed. You seem to be installing Flintrock into your global Python environment, which makes it possible that something else you installed after Flintrock into the same global environment bumped the version of Paramiko down.

If you hit this issue again, feel free to reopen the ticket and we can continue the discussion.