Encoding error - Githubissues

estribiyo commented 5 years ago

LyricsMaster version: 2.7.23
Python version: 3.5
Operating System: Debian Linux 9

Description

Trying to download lyrics from MusicMatch (and others), with special chars (letters with accent mark) or even without it, when the page returns NON ASCII chars, it fails... ERROR providers.py 230: 'ascii' codec can't encode character '\xe9' in position 16: ordinal not in range(128) Traceback (most recent call last): File "/opt/python3-venv/lib/python3.5/site-packages/lyricsmaster-2.7.23-py3.5.egg/lyricsmaster/providers.py", line 228, in get_page req = self.session.request('GET', url) File "/opt/python3-venv/lib/python3.5/site-packages/urllib3/request.py", line 68, in request urlopen_kw) File "/opt/python3-venv/lib/python3.5/site-packages/urllib3/request.py", line 89, in request_encode_url return self.urlopen(method, url, extra_kw) File "/opt/python3-venv/lib/python3.5/site-packages/urllib3/poolmanager.py", line 323, in urlopen response = conn.urlopen(method, u.request_uri, kw) File "/opt/python3-venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/opt/python3-venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line 354, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python3.5/http/client.py", line 1107, in request self._send_request(method, url, body, headers) File "/usr/lib/python3.5/http/client.py", line 1142, in _send_request self.putrequest(method, url, **skips) File "/usr/lib/python3.5/http/client.py", line 984, in putrequest self._output(request.encode('ascii')) UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 16: ordinal not in range(128) WARNING providers.py 232: Unable to download url https://www.musixmatch.com/artist/Andrés-Calamaro

What I Did

from lyricsmaster.providers import MusixMatch
p = MusixMatch()
p.get_lyrics('Andrés Calamaro', album='Andrés', song='Paloma')

SekouD commented 5 years ago

Thanks @estribiyo for submitting this issue.

I will make changes to lyricsmaster to handle unicode characters and push an update during the week.

Cheers,

SekouD

estribiyo commented 5 years ago

I was trying to do it myself, but for fear of touching something that I should not (based on the ignorance of internal functioning) ... I preferred to report the issue.

Many thanks.

SekouD commented 5 years ago

I was trying to do it myself, but for fear of touching something that I should not (based on the ignorance of internal functioning) ... I preferred to report the issue.

Many thanks.

No worries @estribiyo and thanks again for your feedback!

SekouD commented 5 years ago

Hi @estribiyo .

I pushed a fix to your issue in #212 .

I will shortly push a new version of lyricsmaster on pypi so that you can install the new version with a pip install

Thanks for your feedback.

Cheers.

SekouD commented 5 years ago

@estribiyo The new version has been released on pypi

estribiyo commented 5 years ago

Thank you...

Still having some troubles with it... I'll report when I have data enough...

estribiyo commented 5 years ago

Searching for Lyrics of On My Own by Jorge Salán

23-03 19:04    INFO      spider.py  132: Buscando letra de: Jorge Salán - On My Own
23-03 19:04    INFO   providers.py   74: Anonymous requests disabled. The connexion will not be anonymous.
23-03 19:04    INFO   providers.py   74: Anonymous requests disabled. The connexion will not be anonymous.
23-03 19:04   ERROR   providers.py  231: 'ascii' codec can't encode character '\xe1' in position 27: ordinal not in range(128)
Traceback (most recent call last):
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/lyricsmaster/providers.py", line 229, in get_page
    req = self.session.request('GET', url)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/request.py", line 68, in request
    **urlopen_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/request.py", line 89, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/poolmanager.py", line 323, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.5/http/client.py", line 1107, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1142, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/lib/python3.5/http/client.py", line 984, in putrequest
    self._output(request.encode('ascii'))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe1' in position 27: ordinal not in range(128)
23-03 19:04 WARNING   providers.py  233: Unable to download url https://search.azlyrics.com/search.php?q=Jorge+Salán
23-03 19:04   ERROR      spider.py  154: 'NoneType' object has no attribute 'data'
23-03 19:04    INFO   providers.py   74: Anonymous requests disabled. The connexion will not be anonymous.
23-03 19:04    INFO   providers.py   74: Anonymous requests disabled. The connexion will not be anonymous.
23-03 19:04   ERROR   providers.py  231: 'ascii' codec can't encode character '\xe1' in position 43: ordinal not in range(128)
Traceback (most recent call last):
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/lyricsmaster/providers.py", line 229, in get_page
    req = self.session.request('GET', url)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/request.py", line 68, in request
    **urlopen_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/request.py", line 89, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/poolmanager.py", line 323, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/javi/.local/share/virtualenvs/audiodb-gqCRPtuF/lib/python3.5/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.5/http/client.py", line 1107, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1142, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/lib/python3.5/http/client.py", line 984, in putrequest
    self._output(request.encode('ascii'))
UnicodeEncodeError: 'ascii' codec can't encode character '\xe1' in position 43: ordinal not in range(128)
23-03 19:04 WARNING   providers.py  233: Unable to download url https://www.lyrics007.com/search.php?category=artist&q=Jorge+Salán
23-03 19:04   ERROR      spider.py  154: 'NoneType' object has no attribute 'data'
23-03 19:04    INFO   providers.py   74: Anonymous requests disabled. The connexion will not be anonymous.
23-03 19:04    INFO   providers.py  310: Downloading Graffire
23-03 19:04    INFO   providers.py  320: Skipped downloading Graffire as no lyrics matched.
23-03 19:04    INFO     audiodb.py 1092: []

SekouD commented 5 years ago

Hi @estribiyo , I investigqted your issue and found 2 problems:

I forgot to safe quote the url when there are query parameters in the URL
AzLyrics changed the layout of their website, I have to modify the AzLyrics class to handle the changes.

I will push the changes today.

Cheers.

SekouD commented 5 years ago

Hi @estribiyo.

I made the necessary changes to fix the issue you were having with encoding errors on AzLyrics.

I pushed the changes on the dev branch if you want to try them out.

I am currently implementing changes to lyricsmaster because Azlyrics changed their layout and Lyrics007 disabled their search interface. I have found a fix and will release a new version of lyricsmaster during the weekend.

Cheers.

estribiyo commented 5 years ago

I switched to dev branch as suggested, seems to work fine...

SekouD commented 5 years ago

HI @estribiyo .

I just released version 2.8.0 of lyricsmaster.

It fixes the issues with encoding and also updates the way lyricsmaster interfaces with Lyrics007 and AzLyrics.

Thank you for your feedback, your support and your contributions.

Cheers.

SekouD / lyricsmaster

Encoding error #211

Description

What I Did