search

kokarare1212 / librespot-python

Open Source Spotify Client
http://librespot-python.rtfd.io
Apache License 2.0
229 stars 41 forks source link

[BUG] ValueError: invalid literal for int() with base 10 and Max retries exceeded with url #241

Open BluDood opened 11 months ago

BluDood commented 11 months ago

Describe the bug While loading a song to then download it, I seem to randomly get an error saying this: ValueError: invalid literal for int() with base 10: 'Expires=1697802182~FullPath~hmac=Ne8qK1vrknzKhgnDJvCTLDydUxDCxW9' I also seem to get this other error, but thats a lot more rare: (Caused by SSLError(CertificateError("hostname 'audio4-gm-fb.spotifycdn.com' doesn't match 'audio-gm-off.spotifycdn.com'")))

To Reproduce Steps to reproduce the behavior:

  1. Log in to librespot and create a session
  2. Start a new stream using session.content_feeder(), for example 
    stream = session.content_feeder().load(
TrackId.from_base62(some_track_id),
VorbisOnlyAudioQuality(AudioQuality.HIGH),
False,
None
)
  3. Run it a couple times until you see the errors

Expected behavior The stream should start, and get the bytes for downloading.

Logs

Traceback (most recent call last):
  File "<path to python install>\Lib\site-packages\urllib3\connectionpool.py", line 714, in urlopen

    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\urllib3\connectionpool.py", line 403, in _make_request
    self._validate_conn(conn)
  File "<path to python install>\Lib\site-packages\urllib3\connectionpool.py", line 1053, in _validate_conn
    conn.connect()
  File "<path to python install>\Lib\site-packages\urllib3\connection.py", line 472, in connect

    _match_hostname(cert, self.assert_hostname or server_hostname)
  File "<path to python install>\Lib\site-packages\urllib3\connection.py", line 545, in _match_hostname
    match_hostname(cert, asserted_hostname)
  File "<path to python install>\Lib\site-packages\urllib3\util\ssl_match_hostname.py", line 155, in match_hostname
    raise CertificateError("hostname %r doesn't match %r" % (hostname, dnsnames[0]))
urllib3.util.ssl_match_hostname.CertificateError: hostname 'audio4-gm-fb.spotifycdn.com' doesn't match 'audio-gm-off.spotifycdn.com'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<path to python install>\Lib\site-packages\requests\adapters.py", line 486, in send

    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\urllib3\connectionpool.py", line 798, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\urllib3\util\retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='audio4-gm-fb.spotifycdn.com', port=443): Max retries exceeded with url: /audio/a66b2030f9f938fe7f5cff94b8b68ece7c67bf18?Expires=1697802355~FullPath~hmac=MozbpkqHKIJSsljQLrJZ13Pu6hCPPyXf8G2hKNABucA= (Caused by SSLError(CertificateError("hostname 'audio4-gm-fb.spotifycdn.com' doesn't match 'audio-gm-off.spotifycdn.com'")))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<path to script>", line 203, in <module>

    download_song(args.track_id)
  File "<path to script>", line 157, in download_song
    stream = session.content_feeder().load(TrackId.from_base62(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 724, in load
    return self.load_track(playable_id, audio_quality_picker, preload,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 785, in load_track

    return self.load_stream(file, track, None, preload, halt_listener)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 739, in load_stream
    return CdnFeedHelper.load_track(self.__session, track, file,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 339, in load_track
    streamer = session.cdn().stream_file(file, key, url, halt_listener)
               ^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 438, in stream_file
    return CdnManager.Streamer(
           ^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 563, in __init__
    response = self.request(range_start=0,
               ^^^^
^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 617, in request
    response = self.__session.client().get(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\requests\sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\requests\sessions.py", line 587, in request

    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\requests\sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\requests\adapters.py", line 517, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='audio4-gm-fb.spotifycdn.com', port=443): Max retries exceeded with url: /audio/a66b2030f9f938fe7f5cff94b8b68ece7c67bf18?Expires=1697801653~FullPath~hmac=eoe76lWHbt3CTG6gv1-Qz6k916ROMvg-OFqQhWpLV5s= (Caused by SSLError(CertificateError("hostname 'audio4-gm-fb.spotifycdn.com' doesn't match 'audio-gm-off.spotifycdn.com'")))
Traceback (most recent call last):
  File "<path to script>", line 203, in <module>

    download_song(args.track_id)
  File "<path to script>", line 157, in download_song
    stream = session.content_feeder().load(TrackId.from_base62(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 722, in load
    return self.load_track(playable_id, audio_quality_picker, preload,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 783, in load_track
    return self.load_stream(file, track, None, preload, halt_listener)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 737, in load_stream

    return CdnFeedHelper.load_track(self.__session, track, file,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 339, in load_track
    streamer = session.cdn().stream_file(file, key, url, halt_listener)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 442, in stream_file
    CdnManager.CdnUrl(self, file.file_id, url),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 487, in __init__
    self.set_url(url)

  File "<path to python install>\Lib\site-packages\librespot\audio\__init__.py", line 531, in set_url
    self.__expiration = int(token_url.query[:i]) * 1000
                        ^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: 'Expires=1697802182~FullPath~hmac=Ne8qK1vrknzKhgnDJvCTLDydUxDCxW9'

Client Information (please complete the following information):

Additional context It has worked perfecly before, maybe spotify have made some changes?

BluDood commented 11 months ago

Same error occurs on macOS using a Mac Mini M1, however this time it always returns one of the errors

kokarare1212 commented 11 months ago

Perhaps the audio CDN URL specification has changed. I'll look into this in more detail.

Torfek commented 11 months ago

Same problem, it doesn't happen every time. Actually, timestamp is wrong, but who knows why.

kokarare1212 commented 11 months ago

https://github.com/kokarare1212/librespot-python/commit/f97b6b2fe1471373dd643f6ecbe6555f002704e1 There seems to be a similar problem with librespot-java. The code was changed based on this pull request. *The behavior has not been verified yet.

tagdara commented 11 months ago

My follow up comment in the other project still applies - you may still encounter errors when getting a chunk from a server with “audio4” in the name. It’s probably either got a bad cert/missing SAN or it’s a canary for this new format.

KagChi commented 11 months ago

Yeah, the new Expires are working. but we still get missmatch cert.

wizjany commented 11 months ago

For people running into this and finding themselves here, as a local and temporary workaround you can turn off SSL verification (e.g., by inserting client.verify = False here) until the server starts returning the correct certificate. Standard disclaimer that turning off SSL verification is bad etc etc.

kokarare1212 commented 11 months ago

Waiting for more information as there is no positive solution at the moment...

KagChi commented 11 months ago

Maybe can we just exclude audio4-gm-fb.spotifycdn.com for time being? the url cant be accessed through browser too (404)

tagdara commented 11 months ago

@KagChi is correct that the only real solution for the time being is to exclude audio4-gm-fb from the pool.

I made the following temporary change to librespot-java that has been working for several hours today. I'm sure someone can supply an equivalent fix over here.

    @NotNull
    private static HttpUrl getUrl(@NotNull Session session, @NotNull StorageResolveResponse resp) {
        String selectedUrl = resp.getCdnurl(session.random().nextInt(resp.getCdnurlCount()));
        while (selectedUrl.contains("audio4-gm-fb")) {
            LOGGER.warn("getUrl picked CDN with known issues {} (forcing re-selection)", selectedUrl );
            selectedUrl = resp.getCdnurl(session.random().nextInt(resp.getCdnurlCount()));
        }
        return HttpUrl.get(selectedUrl);
        // return HttpUrl.get(resp.getCdnurl(session.random().nextInt(resp.getCdnurlCount())));
    }
kokarare1212 commented 11 months ago

I'll adopt that idea for now. 2bef6c4a5dc2bee5e4d9dc2a070bd5653409f3b9

@KagChi is correct that the only real solution for the time being is to exclude audio4-gm-fb from the pool.

I made the following temporary change to librespot-java that has been working for several hours today. I'm sure someone can supply an equivalent fix over here.

    @NotNull
    private static HttpUrl getUrl(@NotNull Session session, @NotNull StorageResolveResponse resp) {
        String selectedUrl = resp.getCdnurl(session.random().nextInt(resp.getCdnurlCount()));
        while (selectedUrl.contains("audio4-gm-fb")) {
            LOGGER.warn("getUrl picked CDN with known issues {} (forcing re-selection)", selectedUrl );
            selectedUrl = resp.getCdnurl(session.random().nextInt(resp.getCdnurlCount()));
        }
        return HttpUrl.get(selectedUrl);
        // return HttpUrl.get(resp.getCdnurl(session.random().nextInt(resp.getCdnurlCount())));
    }
richteas75 commented 11 months ago

The same issue happens for me as of today, Mon, Nov. 06, 2023 with the host audio-gm-fb.spotifycdn.com

Replacing the line while "audio4-gm-fb" in selected_url: with while "audio4-gm-fb" in selected_url or "audio-gm-fb" in selected_url: helped fix the issue for me today.

Even simply using while "audio-gm-fb" in selected_url: worked today

floodwayprintco commented 6 months ago

f97b6b2 There seems to be a similar problem with librespot-java. The code was changed based on this pull request. *The behavior has not been verified yet.

I use Spocon with Owntone and have been have been trying to track down why it's randomly pausing.

Do you think this issue is the same? https://github.com/spocon/spocon/issues/62

I posted a log and there are some failed CDN things in there.