search

SekouD / lyricsmaster

LyricsMaster is a library for downloading lyrics from multiple lyrics providers.
https://pypi.python.org/pypi/lyricsmaster
MIT License
22 stars 8 forks source link

SSL Error on lyrics007 #214

Closed estribiyo closed 5 years ago

estribiyo commented 5 years ago

Description

Requesting lyrics to Lyrics007 fails because SSL error.

Also, sometimes, the lyrics are incomplete... (about the 200 first chars only), can't specify now if it happens with one or all the providers, sorry. I'll tell you when I have more data about it.

What I Did

Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from lyricsmaster.providers import Lyrics007
>>> p = Lyrics007()
Anonymous requests disabled. The connexion will not be anonymous.
>>> p.get_lyrics('El Inquilino Comunista', album='El Inquilino Comunista', song='Moonwalk')
HTTPSConnectionPool(host='www.lyrics007.com', port=443): Max retries exceeded with url: /search.php?category=artist&q=El+Inquilino+Comunista (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)'),))
Traceback (most recent call last):
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
    conn.connect()
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connection.py", line 344, in connect
    ssl_context=context)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/util/ssl_.py", line 344, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 385, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 760, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 996, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 641, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/lyricsmaster/providers.py", line 229, in get_page
    req = self.session.request('GET', url)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/request.py", line 68, in request
    **urlopen_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/request.py", line 89, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/poolmanager.py", line 323, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.lyrics007.com', port=443): Max retries exceeded with url: /search.php?category=artist&q=El+Inquilino+Comunista (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)'),))
HTTPSConnectionPool(host='www.lyrics007.com', port=443): Max retries exceeded with url: /search.php?category=artist&q=El+Inquilino+Comunista (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)'),))
Traceback (most recent call last):
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
    conn.connect()
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connection.py", line 344, in connect
    ssl_context=context)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/util/ssl_.py", line 344, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 385, in wrap_socket
    _context=self)
  File "/usr/lib/python3.5/ssl.py", line 760, in __init__
    self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 996, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 641, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/lyricsmaster/providers.py", line 229, in get_page
    req = self.session.request('GET', url)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/request.py", line 68, in request
    **urlopen_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/request.py", line 89, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/poolmanager.py", line 323, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 667, in urlopen
    **response_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.lyrics007.com', port=443): Max retries exceeded with url: /search.php?category=artist&q=El+Inquilino+Comunista (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720)'),))
Unable to download url https://www.lyrics007.com/search.php?category=artist&q=El+Inquilino+Comunista
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/lyricsmaster/providers.py", line 285, in get_lyrics
    raw_html = self.get_artist_page(artist)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/lyricsmaster/providers.py", line 246, in get_artist_page
    url = self._make_artist_url(artist)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/lyricsmaster/providers.py", line 875, in _make_artist_url
    return self.search(artist)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/lyricsmaster/providers.py", line 888, in search
    search_results = self.get_page(url).data
AttributeError: 'NoneType' object has no attribute 'data'
SekouD commented 5 years ago

Hi @estribiyo, thanks for submitting the issue.

I will investigate during the week end and keep you updated.

Cheers.

SekouD commented 5 years ago

Hi @estribiyo, I have been investigating your issue but I can not replicate your SSL ERROR.

When I run this code snippet

from lyricsmaster.providers import Lyrics007

p = Lyrics007()

moonwalk = p.get_lyrics('El Inquilino Comunista', album='El Inquilino Comunista', song='Moonwalk')
the_fall = p.get_lyrics('El Inquilino Comunista', album='El Inquilino Comunista', song='The Fall')

I get a Discography object with the right album but no song for monwalk because lyrics007 does not have the lyrics for this song.

When I try with the song The Fall I get back the expected result with this song object:

lyricsmaster.models.Song(The Fall,    --   El Inquilino Comunista, El Inquilino Comunista)

having the following lyrics:

'We will, we will, follow

\'cause i tripped in yours

So, don\'t turn back your manners

\'cause i\'m made in yours

Falling, falling, drinking, drawing

I tripped in yours'

While reading around about ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:720) I found that the issue can be usually fixed by installing/reinstalling the package certifi with:

pip install certifi

I hope it will solve your issue.

Cheers.

SekouD commented 5 years ago

Also, sometimes, the lyrics are incomplete... (about the 200 first chars only), can't specify now if it happens with one or all the providers, sorry. I'll tell you when I have more data about it.

I found a bug in the musixmatch code that sometimes prevented the lyrics to be parsed incorrectly and only parsed the first sentence. I pushed the changes in #217.

The changes are available in lyricsmaster v 2.7.25

estribiyo commented 5 years ago

I think it could be due to a mass lyrics request in a short period of time... so it was my fault. :(

SekouD commented 5 years ago

No worries mate :)

Do not hesitate to open new issues whenever you encounter a problem while using lyricsmaster, this will keep improving the software.

Cheers.