search

dcos / dcos-cli

The command line for DC/OS.
https://docs.d2iq.com/mesosphere/dcos/latest/cli/
Apache License 2.0
224 stars 119 forks source link

HTTP module failing SSL #686

Closed mgummelt closed 5 years ago

mgummelt commented 8 years ago

See https://github.com/mesosphere/dcos-spark/issues/34

ANierbeck commented 8 years ago

Is there any known workaround for this issue? We run into this issue, since upgrading to DC/OS 1.7 ... as 1.6 is EOL ... which is a pity if you see those kind of bugs ...

zutherb commented 8 years ago

same problem

stefan79 commented 8 years ago

Please the put the s back in smack

tamarrow-zz commented 8 years ago

@ANierbeck have you seen this with the binary CLI? I assume this is due to bugs with older versions of python/ssl library.

ANierbeck commented 8 years ago

@tamarrow ... haven't had the chance to verify with the binary CLI yet ... it's on my list :)

vkarpov15 commented 8 years ago

Any workarounds for this? Getting the same issue with kafka after I uninstalled and re-installed

$ dcos --log-level=ERROR package install kafka --cli
Installing CLI subcommand for package [kafka] version [1.1.11-0.10.0.0]
MainThread: 2016-08-29 12:58:14,566 /home/val/Workspace/Booster/dcos/dcos/dcos/local/lib/python2.7/site-packages/dcos/subcommand.py:_execute_command:581 - Command script's stdout: Collecting bin-wrapper==0.0.1 from https://downloads.mesosphere.com/kafka/assets/1.1.11-0.10.0.0/bin_wrapper-0.0.1-py2.py3-none-any.whl (from -r /tmp/tmpLlcPWp (line 1))

MainThread: 2016-08-29 12:58:14,567 /home/val/Workspace/Booster/dcos/dcos/dcos/local/lib/python2.7/site-packages/dcos/subcommand.py:_execute_command:582 - Command script's stderr: /home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Exception:
Traceback (most recent call last):
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/commands/install.py", line 310, in run
    wb.build(autobuilding=True)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/wheel.py", line 750, in build
    self.requirement_set.prepare_files(self.finder)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/req/req_set.py", line 370, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/req/req_set.py", line 587, in _prepare_file
    session=self.session, hashes=hashes)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/download.py", line 810, in unpack_url
    hashes=hashes
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/download.py", line 649, in unpack_http_url
    hashes)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/download.py", line 842, in _download_http_url
    stream=True,
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 487, in get
    return self.request('GET', url, **kwargs)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/download.py", line 378, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/adapter.py", line 46, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/home/val/.dcos/subcommands/kafka/env/local/lib/python2.7/site-packages/pip/_vendor/requests/adapters.py", line 477, in send
    raise SSLError(e, request=request)
SSLError: [Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Error installing 'kafka' package.
Run with `dcos --log-level=ERROR` to see the full output.

And yes, my core.dcos_url uses http://, no https, and core.ssl_verify is false.

tamarrow-zz commented 8 years ago

@vkarpov15 Can you please try with the latest binary: https://github.com/dcos/dcos-cli/releases/tag/0.4.11

ymeymann commented 7 years ago

Having the same problem dcoscli.version=0.4.13 dcos.version=1.8.4 Python 2.7.12

tlahn commented 7 years ago

Same here:

dcos --version

dcoscli.version=0.4.14
dcos.version=1.8.6
dcos.commit=cfccfbf84bbba30e695ae4887b65db44ff216b1d
dcos.bootstrap-id=405172d16eaff8798d6b090dac99b51a8a9004d7

python --version Python 2.7.12

OS: Ubuntu 14.04

dcos config show

core.dcos_acs_token ********
core.dcos_url http://*********
core.ssl_verify false

dcos --log-level=ERROR package install spark --cli

Installing CLI subcommand for package [spark] version [1.0.6-2.0.2]
MainThread: 2016-12-15 13:17:53,813 dcos/subcommand.py:_execute_command:586 - Command script's stdout: b'Collecting dcos-spark==0.5.19 from https://downloads.mesosphere.com/spark/assets/1.0.6-2.0.2/dcos_spark-0.5.19-py2.py3-none-any.whl (from -r /tmp/tmp7ctm4ypi (line 1))
'
MainThread: 2016-12-15 13:17:53,814 dcos/subcommand.py:_execute_command:587 - Command script's stderr: b'/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Exception:
Traceback (most recent call last):
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 659, in unpack_http_url
    hashes)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 853, in _download_http_url
    stream=True,
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 488, in get
    return self.request(\'GET\', url, **kwargs)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/home/dev/.dcos/subcommands/spark/env/local/lib/python2.7/site-packages/pip/_vendor/requests/adapters.py", line 497, in send
    raise SSLError(e, request=request)
SSLError: [Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
'
Error installing 'spark' package.
Run with `dcos --log-level=ERROR` to see the full output.
rupakg commented 7 years ago

Anything on this issue yet its been open for a while. kafka, spark installs practically not usable via CLI

tamarrow-zz commented 7 years ago

@rupakg have you tried with the latest CLI binary?

rupakg commented 7 years ago

@tamarrow I will try 0.4.11. was discouraged by @ymeymann comment that it was not fixed after he got 0.4.13.

tamarrow-zz commented 7 years ago

The difference is the pure python installation and the binary CLI. The pure python installing (especially with python2.7) has bugs, the binary CLI is more stable.

ymeymann commented 7 years ago

Upgrading macOS to Sierra fixed the problem for me.

yuxip commented 7 years ago

@tamarrow I grabbed the binary CLI from https://downloads.dcos.io/binaries/cli/linux/x86-64/dcos-1.8/dcos but I am still having the exact same issue. MainThread: 2017-02-12 18:24:42,541 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_make_request:400 - http://m1.dcos:80 "POST /package/install HTTP/1.1" 200 256

MainThread: 2017-02-12 18:24:46,415 dcos/subcommand.py:_execute_command:589 - Command script's stderr: b'/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this.

File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/requests/adapters.py", line 497, in send\n raise SSLError(e, request=request)\nSSLError: [Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure\n

It looks like cli is using python3.4 but somehow the spark package was still using python2.7 which produced the ssl error. Or perhaps i just completely misunderstood this error.

Any suggestions on how to get around this? Thank you!

tamarrow-zz commented 7 years ago

@yuxip now that you have the binary can you uninstall and reinstall the spark CLI? dcos package uninstall spark --cli and dcos package install spark --cli

yuxip commented 7 years ago

Thanks for your reply @tamarrow! I still have the same problem when I tried to reinstall the spark CLI. Here is the debug-level logs

[yuxi@azure dcos-vagrant]$ dcos --log-level=DEBUG package install spark --cli
MainThread: 2017-02-13 20:51:27,683 dcos/http.py:_request:87 - Sending HTTP ['get'] to ['http://m1.dcos/capabilities']: {'Accept': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1', 'Content-Type': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1'}
MainThread: 2017-02-13 20:51:27,690 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_new_conn:212 - Starting new HTTP connection (1): m1.dcos
MainThread: 2017-02-13 20:51:27,697 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_make_request:400 - http://m1.dcos:80 "GET /capabilities HTTP/1.1" 200 103
MainThread: 2017-02-13 20:51:27,697 dcos/http.py:_request:117 - Received HTTP response [200]: {'Connection': 'keep-alive', 'Content-Length': '103', 'Server': 'openresty/1.9.15.1', 'Date': 'Tue, 14 Feb 2017 04:51:27 GMT', 'Content-Type': 'application/vnd.dcos.capabilities+json;charset=utf-8;version=v1'}
MainThread: 2017-02-13 20:51:27,698 dcos/util.py:timer:533 - duration: dcos.http._request: 0.01s
MainThread: 2017-02-13 20:51:27,698 dcos/http.py:_request:87 - Sending HTTP ['post'] to ['http://m1.dcos/package/describe']: {'Accept': 'application/vnd.dcos.package.describe-response+json;charset=utf-8;version=v2', 'Content-Type': 'application/vnd.dcos.package.describe-request+json;charset=utf-8;version=v1'}
MainThread: 2017-02-13 20:51:27,700 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_new_conn:212 - Starting new HTTP connection (1): m1.dcos
MainThread: 2017-02-13 20:51:27,718 cli/env/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py:_make_request:400 - http://m1.dcos:80 "POST /package/describe HTTP/1.1" 200 15268
MainThread: 2017-02-13 20:51:27,719 dcos/http.py:_request:117 - Received HTTP response [200]: {'Connection': 'keep-alive', 'Content-Length': '15268', 'Server': 'openresty/1.9.15.1', 'Date': 'Tue, 14 Feb 2017 04:51:27 GMT', 'Content-Type': 'application/vnd.dcos.package.describe-response+json;charset=utf-8;version=v2'}
MainThread: 2017-02-13 20:51:27,719 dcos/util.py:timer:533 - duration: dcos.http._request: 0.02s
Installing CLI subcommand for package [spark] version [1.0.7-2.1.0]
MainThread: 2017-02-13 20:51:27,720 dcos/util.py:ensure_dir_exists:129 - Creating directory: '/home/yuxi/.dcos/subcommands/spark'
MainThread: 2017-02-13 20:51:27,720 dcos/subcommand.py:_execute_command:578 - Calling: ['/usr/bin/virtualenv', '--version']
MainThread: 2017-02-13 20:51:27,756 dcos/subcommand.py:_execute_command:591 - Command script's stdout: b'15.1.0\n'
MainThread: 2017-02-13 20:51:27,757 dcos/subcommand.py:_execute_command:592 - Command script's stderr: b''
MainThread: 2017-02-13 20:51:27,757 dcos/subcommand.py:_execute_command:578 - Calling: ['/usr/bin/virtualenv', '/home/yuxi/.dcos/subcommands/spark/env']
MainThread: 2017-02-13 20:51:29,658 dcos/subcommand.py:_execute_command:591 - Command script's stdout: b'New python executable in /home/yuxi/.dcos/subcommands/spark/env/bin/python\nInstalling setuptools, pip, wheel...done.\n'
MainThread: 2017-02-13 20:51:29,658 dcos/subcommand.py:_execute_command:592 - Command script's stderr: b''
MainThread: 2017-02-13 20:51:29,659 dcos/subcommand.py:_execute_command:578 - Calling: ['/home/yuxi/.dcos/subcommands/spark/env/bin/pip', 'install', '--requirement', '/tmp/tmp7x3uafp1']
MainThread: 2017-02-13 20:51:30,111 dcos/subcommand.py:_execute_command:588 - Command script's stdout: b'Collecting dcos-spark==0.5.19 from https://downloads.mesosphere.com/spark/assets/1.0.7-2.1.0/dcos_spark-0.5.19-py2.py3-none-any.whl (from -r /tmp/tmp7x3uafp1 (line 1))\n'
MainThread: 2017-02-13 20:51:30,111 dcos/subcommand.py:_execute_command:589 - Command script's stderr: b'/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.\n  SNIMissingWarning\n/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.\n  InsecurePlatformWarning\nException:\nTraceback (most recent call last):\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main\n    status = self.run(options, args)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/commands/install.py", line 335, in run\n    wb.build(autobuilding=True)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/wheel.py", line 749, in build\n    self.requirement_set.prepare_files(self.finder)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files\n    ignore_dependencies=self.ignore_dependencies))\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/req/req_set.py", line 620, in _prepare_file\n    session=self.session, hashes=hashes)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/download.py", line 821, in unpack_url\n    hashes=hashes\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/download.py", line 659, in unpack_http_url\n    hashes)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/download.py", line 853, in _download_http_url\n    stream=True,\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 488, in get\n    return self.request(\'GET\', url, **kwargs)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/download.py", line 386, in request\n    return super(PipSession, self).request(method, url, *args, **kwargs)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 475, in request\n    resp = self.send(prep, **send_kwargs)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 596, in send\n    r = adapter.send(request, **kwargs)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/cachecontrol/adapter.py", line 47, in send\n    resp = super(CacheControlAdapter, self).send(request, **kw)\n  File "/home/yuxi/.dcos/subcommands/spark/env/lib/python2.7/site-packages/pip/_vendor/requests/adapters.py", line 497, in send\n    raise SSLError(e, request=request)\nSSLError: [Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure\n'
Error installing 'spark' package.
Run with `dcos --log-level=ERROR` to see the full output.

To me it still looks like cli was using python3.4 but the spark subcommand was using python2.7. And the request module in python2.7 had the ssl problem.

ps. my dcos version

[yuxi@azure dcos-vagrant]$ dcos --version
dcoscli.version=0.4.15
dcos.version=1.8.7
dcos.commit=1b43ff7a0b9124db9439299b789f2e2dc3cc086c
dcos.bootstrap-id=e73ba2b1cd17795e4dcb3d6647d11a29b9c35084
yuxip commented 7 years ago

Just to close the loop. The problem is gone after I 'upgraded' from Fedora 21 to Ubuntu 16.04 LTS.

armandgrillet commented 5 years ago

On old versions of macOS, I was able to fix the issue with:

networksetup -setv6off Wi-Fi && networksetup -setv6off "Thunderbolt Ethernet"

Closing this issue as the problem seems to not exist anymore.