Freezes at fetching the lyrics

[Flova]

What is the purpose of your issue?

• [x] Bug
• [ ] Feature Request
• [ ] Question
• [ ] Other

Description

Hi, Sometimes when I try to download a playlist, the tool freezes, while fetching the lyrics. This happens only at a few songs, like one of five. I also had look on the issue #420, but it didn't solve this behavior. Terminating the tool by hand shows the following log.

Thanks Flova

Log

INFO: Checking and removing any duplicate tracks in reading rockmetal-2.txt
INFO: Overriding rockmetal-2.txt with unique tracks
INFO: Preparing to download 124 songs

DEBUG: Fetching metadata for given track URL
DEBUG: Fetching lyrics
^CERROR:
    Traceback (most recent call last):
      File "/usr/lib/python3.7/site-packages/spotdl/spotdl.py", line 62, in main
        match_args()
      File "/usr/lib/python3.7/site-packages/spotdl/spotdl.py", line 38, in match_args
        list_dl.download_list()
      File "/usr/lib/python3.7/site-packages/spotdl/downloader.py", line 196, in download_list
        return self._download_list()
      File "/usr/lib/python3.7/site-packages/spotdl/downloader.py", line 204, in _download_list
        track_dl = Downloader(raw_song, number=number)
      File "/usr/lib/python3.7/site-packages/spotdl/downloader.py", line 93, in __init__
        self.content, self.meta_tags = youtube_tools.match_video_and_metadata(raw_song)
      File "/usr/lib/python3.7/site-packages/spotdl/youtube_tools.py", line 55, in match_video_and_metadata
        meta_tags = spotify_tools.generate_metadata(track)
      File "/usr/lib/python3.7/site-packages/spotdl/spotify_tools.py", line 74, in generate_metadata
        meta_tags["artists"][0]["name"], meta_tags["name"]
      File "/home/florian/.local/lib/python3.7/site-packages/lyricwikia/lyricwikia.py", line 30, in get_lyrics
        return get_all_lyrics(artist, song, linesep, timeout)[0]
      File "/home/florian/.local/lib/python3.7/site-packages/lyricwikia/lyricwikia.py", line 36, in get_all_lyrics
        response = _requests.get(url, timeout=timeout)
      File "/usr/lib/python3.7/site-packages/requests/api.py", line 75, in get
        return request('get', url, params=params, **kwargs)
      File "/usr/lib/python3.7/site-packages/requests/api.py", line 60, in request
        return session.request(method=method, url=url, **kwargs)
      File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
        r = adapter.send(request, **kwargs)
      File "/usr/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
        timeout=timeout
      File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
        chunked=chunked)
      File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
        self._validate_conn(conn)
      File "/usr/lib/python3.7/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
        conn.connect()
      File "/usr/lib/python3.7/site-packages/urllib3/connection.py", line 344, in connect
        ssl_context=context)
      File "/usr/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 344, in ssl_wrap_socket
        return context.wrap_socket(sock, server_hostname=server_hostname)
      File "/usr/lib/python3.7/ssl.py", line 412, in wrap_socket
        session=session
      File "/usr/lib/python3.7/ssl.py", line 853, in _create
        self.do_handshake()
      File "/usr/lib/python3.7/ssl.py", line 1117, in do_handshake
        self._sslobj.do_handshake()
    KeyboardInterrupt

[ritiek]

I also had look on the issue #420, but it didn't solve this behavior.

So are you IPv4 only? I'd like you to confirm whether you can reach IPv6 sites here https://test-ipv6.com/, just to be sure.

[Flova]

My connection supports full dual stack (confirmed with your link), with IPv4 and IPv6. After i read your first answer to #420 i tried to set IPv4 as preferred protocol, which didn't solve the Problem. A interesting behavior is, that after a restart the tool downloads the next few songs (including the ones that previously stuck) before stopping again. Quite obvious but mentionable: It hangs at self._sslobj.do_handshake() every time I terminate it.

[ritiek]

I see. I don't know much about how IPv6 works and there's nothing I can test IPv6 with at the moment.

So, from the log it gets stuck while making an SSL handshake with http://lyrics.wikia.com. Lyricwikia python library uses HTTPS but since the website itself doesn't support HTTPS, changing it to use HTTP shouldn't make a difference.

That said, this Python code would probably behave the same (getting stuck 1/5 times). Could you try it out:

import lyricwikia

# some songs to test against
lyricwikia.get_lyrics("Alan Walker", "Faded")
lyricwikia.get_lyrics("Against_The_Current", "Gravity")
lyricwikia.get_lyrics("Porter Robinson & Madeon", "Shelter")
lyricwikia.get_lyrics("Marshmello & Bastille", "Happier")

And what if you now replace the url to use HTTP:

import lyricwikia
lyricwikia.lyricwikia.__BASE_URL__  = "http://lyrics.wikia.com"

# some songs to test against
lyricwikia.get_lyrics("Alan Walker", "Faded")
lyricwikia.get_lyrics("Against_The_Current", "Gravity")
lyricwikia.get_lyrics("Porter Robinson & Madeon", "Shelter")
lyricwikia.get_lyrics("Marshmello & Bastille", "Happier")

Does it still get stuck with HTTP?

[Flova]

Sorry i had some busy days. Maybe i am able to test it today.

[Flova]

I just tested the lyricwikia package. Ironically both tests worked absolutely fine... It downloaded the Lyrics via HTTPS and HTTP, while the same song at the same time hang in the playlist download.

[ritiek]

Dats weird. Let's try using HTTP in the code anyway. See if adding this line:

lyricwikia.lyricwikia.__BASE_URL__  = "http://lyrics.wikia.com"

just after the import lyricwikia line here: https://github.com/ritiek/spotify-downloader/blob/b72eb773f3981d0f27b85792002845ecf0b69911/spotdl/spotify_tools.py#L3-L4

makes any difference.

[Flova]

I tried it and the same error occurs. Now I deactivated the Lyrics manually in the code. Should I maybe add an config parameter for it?