SSL error on track lookup

[Hillshum]

I'm running on Ubuntu 15.04, with Python 2.7.9. I can log in to both services, identify playlists that I want, and then Pyportify runs through each selected playlist. It never increments any of the counters for found, Karaoke, etc, and in the console I can see it gets an SSLError for each track in the playlist. Once each playlist has been processed (with failure for each track), Pyportify finishes.

Here's the traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pyportify/views.py", line 207, in search_gm_track
    track = g.find_best_track(search_query)
  File "/usr/local/lib/python2.7/dist-packages/pyportify/google.py", line 67, in find_best_track
    data = self.search_all_access(search_query)
  File "/usr/local/lib/python2.7/dist-packages/pyportify/google.py", line 63, in search_all_access
    data = self._http_get(url)
  File "/usr/local/lib/python2.7/dist-packages/pyportify/google.py", line 95, in _http_get
    headers=headers
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 184, in get
    return self.request(METHOD_GET, request_uri, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 157, in request
    sock = self._connection_pool.get_socket()
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 133, in get_socket
    return self._create_socket()
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 116, in _create_socket
    raise first_error
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
<Greenlet at 0x7f1bb9888690: search_gm_track((10, u'spotify:user:hillshum:playlist:0b8pEDh8pdAk)> failed with SSLError

That happens with every single track in my library, whether my own playlist or one from someone else. I've found things like http://stackoverflow.com/questions/27835619/ssl-certificate-verify-failed-error, but Pyportify does its own ssl-tweaking and I'm not sure what the best way to fix things is. Still, some sort of bug exists.

[jwannebo]

Confirming this issue on a Linux Mint VM running in Windows, same SSL error, which happens ONLY when pyportify tries to search for the song on Google Play. Logging in to Google, and getting all the tracks from Spotify (judging by the console spam) both work beautifully.

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 327, in run
    result = self._run(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pyportify/views.py", line 207, in search_gm_track
    track = g.find_best_track(search_query)
  File "/usr/local/lib/python2.7/dist-packages/pyportify/google.py", line 67, in find_best_track
    data = self.search_all_access(search_query)
  File "/usr/local/lib/python2.7/dist-packages/pyportify/google.py", line 63, in search_all_access
    data = self._http_get(url)
  File "/usr/local/lib/python2.7/dist-packages/pyportify/google.py", line 95, in _http_get
    headers=headers
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 184, in get
    return self.request(METHOD_GET, request_uri, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 157, in request
    sock = self._connection_pool.get_socket()
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 133, in get_socket
    return self._create_socket()
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 116, in _create_socket
    raise first_error
SSLError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
<Greenlet at 0x7f9a01322eb0: search_gm_track((296, u'spotify:user:None:playlist:starred', 'Lond)> failed with SSLError

[FaFassou]

I can confirm, I get the exact same traceback Hillshum posted.

[robincroneld]

I have the exact same problem when trying to transfer playlists from Spotify to google music. :( I hope this is fixable! I really want to leave spotify.

[adrianrudnik]

As a very insecure and dirty fix:

Edit /usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py

Around line 180 should be

        default_options = {
            'ciphers': _DEFAULT_CIPHERS,
            'ca_certs': _CA_CERTS,
            'cert_reqs': gevent.ssl.CERT_NONE
        }

and in /usr/local/lib/python2.7/dist-packages/backports/ssl_match_hostname/__init__.py around line 64 just add a return to skip the whole match_hostname function like

    return
    if not cert:
        raise ValueError("empty or no certificate")

• DO backups beforehand.
• DO NOT use this on any system that uses python for other critical or production like stuff.
• DO revert back to backups, otherwise the system will be compromised.

[robincroneld]

I cannot belive that worked! Thanks a bunch kreischweide! Im restoring the files just as soon as i have moved all my playlists. Thanks again!

[CPA-Poke]

@kreischweide, this is my first go at Python and I'm not very familiar with function logic. Would you or @robincroneld mind sharing the finished product(s) from each file after making your edits?

[Hillshum]

Just upgraded to Ubuntu 15.10 (with Python 2.7.10) and things work fine.

[rckclmbr]

I believe I have this resolved in https://github.com/rckclmbr/pyportify/commit/6a5e49bbe554c2cc1a517ea3569acf17c6ed2f48 (pyportify v0.2.8). Is anyone willing to give it a try?

[CPA-Poke]

I updated, but still getting the same error:

http://pastebin.com/32FqGN5j

Really do appreciate you taking the time to continue supporting this!

[robincroneld]

Here are the files with the modifications. Dont forget to backup your old files so you can restore it after you are done. https://www.dropbox.com/s/d5wsmtn2wth2fon/connectionpool.py?dl=0 <--- connectionpool.py Which goes to: /usr/local/lib/python2.7/dist-packages/geventhttpclient/

https://www.dropbox.com/s/l5zvo46ylel1fsb/__init__.py?dl=0 <--- init.py which goes to: /usr/local/lib/python2.7/dist-packages/backports/ssl_match_hostname/

I rebooted after i modified the files before i tested if everything worked. Dont know if thats required, but thats what i did.

Tested in Linux Mint 17.2

[robincroneld]

@rckclmbr i would gladly try, but im not sure where all the files are supposed to go.

[rckclmbr]

@robincroneld just run pip install --upgrade pyportify==0.2.9 (with sudo if you used it), then try it again.

[rckclmbr]

@CPA-Poke what version of OpenSSL are you using? (openssl version)

[CPA-Poke]

1.0.1f

Have you pushed the update to your download server yet? Got an error trying to update to 0.2.9 just now, but I'm guessing I just tried too early haha

[rckclmbr]

Yep, you're too fast! Should be up now give it a try On Dec 9, 2015 7:28 AM, "Brock Shahan" notifications@github.com wrote:

1.0.1f

Have you pushed the update to your download server yet? Got an error trying to update to 0.2.9 just now, but I'm guessing I just tried too early haha

— Reply to this email directly or view it on GitHub https://github.com/rckclmbr/pyportify/issues/49#issuecomment-163291965.

[CPA-Poke]

Just did, and got a build error. I'm doing this from my phone (remote-ing in to my PC at home) so I just briefly looked at this log, but it does look like the build error had something to do with OpenSSL?

pastebin.com/eHARDXyJ

[CPA-Poke]

So I decided to start with a fresh install and give it another shot (using 0.2.9)...install went fine, and so did the playlist import! Didn't need to apply any of the above hotfixes or anything, just fresh out of the box.

[CPA-Poke]

Whoops, not quite "fresh out of the box," I forgot that I did have to downgrade Flash Socket to 0.5.0 as noted here: https://github.com/rckclmbr/pyportify/issues/47

But I've had to do it that way from the beginning, so nothing different on my fresh install of 0.2.9.

[rckclmbr]

Removed dependency of Flask-socketio, this should be resolved.