search

nchammas / flintrock

A command-line tool for launching Apache Spark clusters.
Apache License 2.0
636 stars 116 forks source link

paramiko: CryptographyDeprecationWarning #295

Closed mdgreenwald closed 4 years ago

mdgreenwald commented 4 years ago

When I use the following command to launch a cluster:

flintrock launch test-cluster \
    --num-slaves 1 \
    --ec2-vpc-id vpc-1234abcd \
    --ec2-subnet-id subnet-1234abcd \
    --spark-version 2.4.4 \
    --ec2-key-name key-name \
    --ec2-identity-file /path/to/key.pem \
    --ec2-ami ami-00068cd7555f543d5 \
    --ec2-user ec2-user

I get the warning message from paramiko below:

Warning: Downloading Spark from an Apache mirror. Apache mirrors are often slow and unreliable, and typically only serve the most recent releases. We strongly recommend you specify a custom download source. For more background on this issue, please see: https://github.com/nchammas/flintrock/issues/238
Launching 2 instances...
/Users/flastname/.pyenv/versions/3.7.2/lib/python3.7/site-packages/paramiko/ecdsakey.py:164: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.ecdsa_curve.curve_class(), pointinfo
/Users/flastname/.pyenv/versions/3.7.2/lib/python3.7/site-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  m.add_string(self.Q_C.public_numbers().encode_point())
/Users/flastname/.pyenv/versions/3.7.2/lib/python3.7/site-packages/paramiko/kex_ecdh_nist.py:96: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
  self.curve, Q_S_bytes
/Users/flastname/.pyenv/versions/3.7.2/lib/python3.7/site-packages/paramiko/kex_ecdh_nist.py:111: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
  hm.add_string(self.Q_C.public_numbers().encode_point())
[54.208.42.197] SSH online.
[3.89.158.142] SSH online.

It appears that this issue is addressed in paramiko v2.5.0. {[1],[2]}

It also appears that if the version of paramiko that flintrock requires were bumped to 2.5.0 that this issue might be resolved.

nchammas commented 4 years ago

Can you try running Flintrock off master? I believe we're already using a newer version of Paramiko.

mdgreenwald commented 4 years ago

You are correct, I did notice that after opening my issue. But it is admittedly nice to be able to work with a specific release version (v0.11.0) rather than a commit. That being said, I could compile a binary from master and circulate it internally at my company.

Thanks for responding!

nchammas commented 4 years ago

I will push a new release over the holidays just to bump the dependencies and maybe also make some minor tweaks. Definitely agree that it's cleaner to work with a proper release.

mdgreenwald commented 4 years ago

Thank you! 🙏🏻

mdgreenwald commented 4 years ago

@nchammas Any update on the new release? :-)

nchammas commented 4 years ago

Follow #300. ;)