Yotta (0.12.2) crashes with OpenSSL.SSL.ZeroReturnError during "yt up"

[teetak01]

I have been getting this kind of crashes quite regularly with several revisions of yotta. Yotta seems to crash everything something new is to be updated. Sometimes I have to repeat "yt up" several times before it goes through without crash.

info: Starting new HTTPS connection (1): codeload.github.com
Fatal Exception, yotta=0.12.2
Traceback (most recent call last):
  File "/usr/local/bin/yt", line 4, in <module>
    yotta.main()
  File "/usr/local/lib/python2.7/dist-packages/yotta/main.py", line 205, in main
    status = args.command(args, following_args)
  File "/usr/local/lib/python2.7/dist-packages/yotta/build.py", line 46, in execCommand
    status = installAndBuild(args, following_args)
  File "/usr/local/lib/python2.7/dist-packages/yotta/build.py", line 98, in installAndBuild
    install_status = install.execCommand(args, [])
  File "/usr/local/lib/python2.7/dist-packages/yotta/install.py", line 62, in execCommand
    return installDeps(args, c)
  File "/usr/local/lib/python2.7/dist-packages/yotta/install.py", line 108, in installDeps
    test = {'own':'toplevel', 'all':True, 'none':False}[args.install_test_deps]
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/component.py", line 565, in satisfyDependenciesRecursive
    test = test
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/component.py", line 349, in __getDependenciesRecursiveWithProvider
    test = test
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/component.py", line 249, in __getDependenciesWithProvider
    satisfyDep, specs
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/component.py", line 231, in satisfyDep
    self.getName()
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/component.py", line 551, in provider
    r = access.satisfyVersionByInstalling(dspec.name, dspec.version_req, self.modulesPath())
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/access.py", line 297, in satisfyVersionByInstalling
    name, version_required, install_into, v, type=type
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/access.py", line 308, in _satisfyVersionByInstallingVersion
    version.unpackInto(working_directory)
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/github_access.py", line 204, in unpackInto
    self.url, directory, self.cache_key, origin_info=self.origin_info
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/github_access.py", line 80, in wrapped
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/github_access.py", line 172, in _getTarball
    origin_info = origin_info
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/access_common.py", line 118, in wrapped
    r = fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/access_common.py", line 282, in unpackTarballStream
    new_cache_key = downloadToCache(stream, hash, cache_key, origin_info)
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/access_common.py", line 237, in downloadToCache
    for chunk in stream.iter_content(4096):
  File "/usr/lib/python2.7/dist-packages/requests/models.py", line 616, in generate
    decode_content=True):
  File "/usr/lib/python2.7/dist-packages/urllib3/response.py", line 225, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/lib/python2.7/dist-packages/urllib3/response.py", line 174, in read
    data = self._fp.read(amt)
  File "/usr/lib/python2.7/httplib.py", line 573, in read
    s = self.fp.read(amt)
  File "/usr/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 152, in read
    data = self._sock.recv(left)
OpenSSL.SSL.ZeroReturnError

[autopulated]

Thanks for reporting this!

Please could you share:

• your python version (python --version)
• your python installed package versions (pip list)
• and your OS version (linux flavour and version)

[teetak01]

Python 2.7.6

adium-theme-ubuntu (0.3.4) apt-xapian-index (0.45) argcomplete (1.0.0) backports.ssl-match-hostname (3.4.0.2) ccsm (0.9.11.3) certifi (2015.9.6.2) cffi (1.4.2) chardet (2.0.1) click (5.1) CoAPthon (2.0.0) colorama (0.3.5) command-not-found (0.3) compizconfig-python (0.9.11.3) cryptography (1.1.2) debtagshw (0.1) defer (1.0.6) dirspec (13.10) docutils (0.12) duplicity (0.6.23) enum34 (1.1.2) functools32 (3.2.3.post2) futures (2.1.6) gcovr (3.2) hgapi (1.7.2) httplib2 (0.8) idna (2.0) intelhex (2.0) ipaddress (1.0.15) Jinja2 (2.8) jsonmerge (1.1.0) jsonpointer (1.10) jsonschema (2.5.1) livereload (2.4.0) lockfile (0.8) Logbook (0.11.3) lxml (3.3.3) Markdown (2.6.3) MarkupSafe (0.23) mbed-clitest (0.0.2) mbed-ls (0.1.22) mbed-test-wrapper (0.0.2) mkdocs (0.14.0) ndg-httpsclient (0.4.0) oauthlib (0.6.1) oneconf (0.3.7.14.4.1) PAM (0.4.2) pathlib (1.0.1) pexpect (3.1) Pillow (2.3.0) pip (7.1.2) piston-mini-client (0.7.5) prettytable (0.7.2) project-generator (0.8.3) project-generator-definitions (0.2.2) psutil (3.2.2) py (1.4.30) pyasn1 (0.1.9) pycparser (2.14) pycrypto (2.6.1) pycups (1.9.66) pyelftools (0.23) PyGithub (1.26.0) Pygments (2.0.2) pygobject (3.12.0) PyJWT (1.4.0) pyOCD (0.6.1) pyOpenSSL (0.15.1) pyserial (2.6) pyshark (0.3.6) pysmbc (1.0.14.1) python-apt (0.9.3.5ubuntu2) python-debian (0.1.21-nmu2ubuntu2) pyusb (1.0.0rc1) pyxdg (0.25) PyYAML (3.11) reportlab (3.0) requests (2.9.1) semantic-version (2.4.2) sessioninstaller (0.0.0) setuptools (19.2) six (1.10.0) software-center-aptd-plugins (0.0.0) Sphinx (1.2.2) system-service (0.1.6) tornado (4.2.1) trollius (2.0) Twisted (15.4.0) Twisted-Core (13.2.0) Twisted-Web (13.2.0) unity-lens-photos (1.0) urllib3 (1.7.1) valinor (0.0.12) vboxapi (1.0) virtualenv (13.1.2) wheel (0.26.0) xdiagnose (3.6.3build2) xmltodict (0.9.2) yattag (1.5.2) yotta (0.12.2) zope.interface (4.0.5)

Linux 3.19.0-41-generic #46~14.04.2-Ubuntu SMP Tue Dec 8 17:46:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux --running inside VirtualBox 4.3.20.r96997

[autopulated]

Thanks!

I think that updating your urllib3 (you have 1.7.1 vs latest 1.13.1) might fix the problem – you should be able to do this with pip install -U urllib3.

If you have trouble updating this due to dependencies on it by other things in your system, you could also consider installing yotta in a virtualenv, which should let you use up-to-date versions of everything.

[teetak01]

Hi, I updated urllib3 to 1.14 with pip, but I am still getting the same crash. I added a bit more info from the beginning.

yt up info: get versions for x86-linux-native-coverage info: Starting new HTTPS connection (1): registry.yottabuild.org info: get versions for mbed-client info: Starting new HTTPS connection (1): registry.yottabuild.org info: get versions for mbed-client-cli warning: Github repository "ARMmbed/mbed-client-cli" has no tagged versions, default branch will be used info: update outdated: mbed-client-cli@0.0.1 -> mbed-client-cli@ from GitHub ARMmbed/mbed-client-cli info: download mbed-client-cli@ from GitHub ARMmbed/mbed-client-cli info: Starting new HTTPS connection (1): codeload.github.com Fatal Exception, yotta=0.12.2 Traceback (most recent call last): File "/usr/local/bin/yt", line 4, in yotta.main() File "/usr/local/lib/python2.7/dist-packages/yotta/main.py", line 205, in main status = args.command(args, following_args) ...continuing exactly the same as above.

The issue seems to come often/always after yotta first prints "info: update outdated:"

[autopulated]

Hmm OK. Now all your relevant seem to be up to date, which is interesting as supposedly this bug is fixed in current versions of requests, and urllib3.

It is worth seeing if:

pip install -U requests[security]

Causes anything to be updated (but I don't think it will).

[teetak01]

All up to date.

[autopulated]

Could you paste the portion of the backtrace corresponding to the part below, but for the updated version of urllib3?

  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/access_common.py", line 237, in downloadToCache
    for chunk in stream.iter_content(4096):
  File "/usr/lib/python2.7/dist-packages/requests/models.py", line 616, in generate
    decode_content=True):
  File "/usr/lib/python2.7/dist-packages/urllib3/response.py", line 225, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/usr/lib/python2.7/dist-packages/urllib3/response.py", line 174, in read
    data = self._fp.read(amt)
  File "/usr/lib/python2.7/httplib.py", line 573, in read
    s = self.fp.read(amt)
  File "/usr/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 152, in read
    data = self._sock.recv(left)

[teetak01]

Strange, I am completely unable to replicate the bug anymore. I was still getting crashes yesterday. Only differences I can think of is rebooting the virtual machine (unlikely) or the fact that my account settings in github were altered yesterday. It well might have been some attempt to update some parts of previously inaccesible private repos which then caused yotta to fail?

[autopulated]

The zero return "error" is actually an indication of success, so I don't think it would have been caused by an error on Github – possibly it required a reboot for the new version of urllib3 to be picked up by yotta for some reason, but I don't know why that would be.

I suppose I will close this issue for now then: if anyone ever experiences the same problem again, with an up-to-date version of urllib3 please do comment/re-open!