johnwmillr / LyricsGenius

Download song lyrics and metadata from Genius.com 🎶🎤
http://www.johnwmillr.com/scraping-genius-lyrics/
MIT License
892 stars 158 forks source link

"HTTPSConnectionPool(host='genius.com', port=443): Read timed out. (read timeout=5)" Error #196

Closed Kyumagi closed 3 years ago

Kyumagi commented 3 years ago

Hello! I tried to search_artist "eminem" but i got the "HTTPSConnectionPool(host='genius.com', port=443): Read timed out. (read timeout=5)" error. I tried to search "drake" and "Sia" instead of "eminem" but got this error again. here is my code and the error:

import lyricsgenius
TOKEN = "MY_ACCESS_TOKEN"
genius = lyricsgenius.Genius(TOKEN)
artist = genius.search_artist("eminem", max_songs=10, sort="title")
Searching for songs by eminem...

Traceback (most recent call last):
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 1010, in _validate_conn
    conn.connect()
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connection.py", line 411, in connect
    self.sock = ssl_wrap_socket(
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\util\ssl_.py", line 428, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\util\ssl_.py", line 472, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "~\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 500, in wrap_socket
    return self.sslsocket_class._create(
File "~\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1040, in _create
    self.do_handshake()
File "~\AppData\Local\Programs\Python\Python38-32\lib\ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
socket.timeout: _ssl.c:1091: The handshake operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen
    retries = retries.increment(
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\util\retry.py", line 531, in increment
    raise six.reraise(type(error), error, _stacktrace)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\packages\six.py", line 735, in reraise
    raise value
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 385, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='genius.com', port=443): Read timed out. (read timeout=5)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\lyricsgenius\api\base.py", line 75, in _make_request
    response = self._session.request(method, uri,
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='genius.com', port=443): Read timed out. (read timeout=5)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "<pyshell#5>", line 1, in <module>
    artist = genius.search_artist("eminem", max_songs=10, sort="title")
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\lyricsgenius\genius.py", line 522, in search_artist
    artist_id = artist_id if artist_id else find_artist_id(artist_name)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\lyricsgenius\genius.py", line 507, in find_artist_id
    response = self.search_all(search_term)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\lyricsgenius\api\public_methods\search.py", line 210, in search_all
    return self.search(search_term, per_page, page, endpoint)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\lyricsgenius\api\public_methods\search.py", line 45, in search
    return self._make_request(path, params_=params, public_api=True)
File "~\AppData\Local\Programs\Python\Python38-32\lib\site-packages\lyricsgenius\api\base.py", line 84, in _make_request
    raise Timeout(error)
requests.exceptions.Timeout: Request timed out:
HTTPSConnectionPool(host='genius.com', port=443): Read timed out. (read timeout=5)

Please help me fix this error. Sorry for my bad english, I'm not a native speaker.

allerter commented 3 years ago

This error doesn't have anything to do with the artist your search. It means that your device couldn't read the data from Genius in time. If you're on version 3, I suggest you try this configuration:

genius = Genius(ACCESS_TOKEN, timeout=15, retries=3)

If you still get that error after setting this configuration, try these methods and see if they give you the timeout error as well:

song = genius.song(1)
artist = genius.artist(1)

If they do, it means something's wrong with your connection to Genius. You should investigate that and maybe try using a proxy. If they don't, it means that something goes wrong when your computer tries to get data from Genius Web API (genius.com/api). This API isn't officially supported and all I can suggest is using a proxy.

On a separate note, make sure to mask your access token when posting your code. I removed it from your question, but you shouldn't use that anymore and generate a new one.

Kyumagi commented 3 years ago

Thanks for your help; I used genius = Genius(ACCESS_TOKEN, timeout=15, retries=3) in my script. but sometimes it works and sometimes it doesn't. I think i should use a proxy as you said(I don't know how to use a proxy please tell me how :D). I tried song = genius.song(1) and artist = genius.artist(1) and they worked well. And... that wasn't exactly my API; I had changed a few of characters before posting. thanks anyway <3.

allerter commented 3 years ago

I usually use a VPN when dealing with Genius. You could do this too, but you should beware that Genius has captcha enabled and requests from some VPN servers hit that captcha. I don't know if that's also the case for proxies, but there are a few ways to use proxies when using the reuqests package. Although it's already been explained by the Requests documentation and in StackOverflow answers, I'll explain briefly how to do it with LyricsGenius:

Using env variables

import os
import lyricsgenius as lg

os.environ['HTTP_PROXY'] = os.environ['http_proxy'] = 'http://http-connect-proxy:3128/'
os.environ['HTTPS_PROXY'] = os.environ['https_proxy'] = 'http://http-connect-proxy:3128/'
os.environ['NO_PROXY'] = os.environ['no_proxy'] = '127.0.0.1,localhost,.local'

genius = lg.Genius(ACCESS_TOKEN, ...)

Adding proxies to the Session

import lyricsgenius as lg

proxies = {
    'http': 'http://user:pass@10.10.1.0:3128',
    'https': 'https://user:pass@10.10.1.0:3128',
}
genius = lg.Genius(ACCESS_TOKEN, ...)
genius._session.proxies = proxies

If you need further information about using proxies, please refer to the links above.