search

spotipy-dev / spotipy

A light weight Python library for the Spotify Web API
http://spotipy.readthedocs.org
MIT License
5.04k stars 959 forks source link

`500` error code in `sp.audio_features()` reported as `429` #805

Open jkulak opened 2 years ago

jkulak commented 2 years ago

Describe the bug 500 error code in sp.audio_features() reported as 429

Your code For this particular ID, https://developer.spotify.com/console/get-audio-features-several-tracks/ returns 500.

audio_features = sp.audio_features(tracks=["1tkkrLY3upIwNfLs4ei8Bh"])

Spotipy returns: 

 "exception": "SpotifyException(429, -1, '/v1/audio-features/?ids=1tkkrLY3upIwNfLs4ei8Bh:\\n Max Retries')"

Expected behavior I would expect Spotipy to show me that it is a 500 error code. So I could debug and verify which ID is causing the issue on Spotify's side. When I am getting 429 I am assuming it is a limit error.

Environment:

Additional context The request exception is caught here: https://github.com/plamere/spotipy/blob/9a627e88f422927822ce39ae9919cc7ab9813dde/spotipy/client.py#L274

And then it is raising 429 here: https://github.com/plamere/spotipy/blob/9a627e88f422927822ce39ae9919cc7ab9813dde/spotipy/client.py#L281

jkulak commented 2 years ago

Btw. those ids also return 500 from Spotify's API for audio_features.

(It is an ASMR album with ultrasonic sounds)

Peter-Schorn commented 2 years ago

What makes you so sure that 429 is the wrong error? How do you know that your requests aren't getting rate-limited by Spotify?

Have you used the debugger to determine that spotipy returns a rate-limited error even when the raw network response from the requests module contains a 500 status code?

jkulak commented 2 years ago

Hey. I checked what error code is returned when requesting same url using same token using Spotify’s console and then also using Postman. It returns 500 in both cases.

Spotipy returns 429 (same token, same url, before and after I tested in console).

It’s super easy to check using code samples and IDs I provided.

dasbts commented 2 years ago

I got the same issue and error (max retries) when feeding a list of songs, and when searching for it (by link in browser and the URI ofc.) it said it was unavailable, however it was not a local song and I got the song info for it when requesting a playlist of songs. My thought is it might just not be available in your market.

My solution after trying multiple fixes is to just if you want to also fetch other songs, do a try except, and just ignore the one that doesn't work.

It's not rate limiting, at least not in my case, because all songs except some few ones don't work, and the try except allows me to fetch multiple other songs and just skip these that are retrying.

Peter-Schorn commented 2 years ago

I was unable to reproduce the rate-limited error with any of the ids although I did get a 500 error with all of these ids. I don't know why this error is returned for certain tracks, but it's clear that this is not due to an issue on spotipy's end because I get the same error when I make the request in Postman. You'll have to ask about this on the Spotify Developer Forums.

Spotipy returns 429 (same token, same url, before and after I tested in console).

Rate limiting is applied on a per-developer application basis, not a per-access token basis, so try creating a new developer application (which means you'll get a new client id and client secret) and then use this application with spotipy only for a single request. Can you still reproduce the rate-limited error?

jkulak commented 2 years ago

Thank you for your answers, I appreciate your time.

You'll have to ask about this on the Spotify Developer Forums.

Yes. But I don't mind 500s from Spotify for some IDs - it happens, it is ok. But I would like Spotipy to show me it was a 500 on Spotify's end and not 429.

This code (assuming you have your env SPOTIPY_CLIENT_ID and SPOTIPY_CLIENT_SECRET)

import spotipy
from spotipy.oauth2 import SpotifyClientCredentials

sp = spotipy.Spotify(auth_manager=SpotifyClientCredentials())
result = sp.audio_features(tracks=["1tkkrLY3upIwNfLs4ei8Bh"])

Results in urllib3 exception, caught and passed byrequests exception and then caught by Spotipy and then reported as spotipy.exceptions.SpotifyException: http status: 429. Which (in my opinion) is not correct, becase my rate limits are OK, and Spotify returns 500 for this particular request (albo same market etc.). Full Traceback (Pastebin for convenience, line 42, https://pastebin.com/1SHAQJbH):

(app) /app # python 500_test.py
Max Retries reached
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/requests/adapters.py", line 440, in send
    resp = conn.urlopen(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/connectionpool.py", line 876, in urlopen
    return self.urlopen(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/connectionpool.py", line 876, in urlopen
    return self.urlopen(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/connectionpool.py", line 876, in urlopen
    return self.urlopen(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/connectionpool.py", line 866, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/audio-features/?ids=1tkkrLY3upIwNfLs4ei8Bh (Caused by ResponseError('too many 500 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/spotipy/client.py", line 240, in _internal_call
    response = self._session.request(
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/requests/sessions.py", line 645, in send
    r = adapter.send(request, **kwargs)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/requests/adapters.py", line 510, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/audio-features/?ids=1tkkrLY3upIwNfLs4ei8Bh (Caused by ResponseError('too many 500 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/500_test.py", line 5, in <module>
    result = sp.audio_features(tracks=["1tkkrLY3upIwNfLs4ei8Bh"])
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/spotipy/client.py", line 1683, in audio_features
    results = self._get("audio-features/?ids=" + ",".join(tlist))
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/spotipy/client.py", line 297, in _get
    return self._internal_call("GET", url, payload, kwargs)
  File "/root/.local/share/virtualenvs/app-4PlAip0Q/lib/python3.10/site-packages/spotipy/client.py", line 281, in _internal_call
    raise SpotifyException(
spotipy.exceptions.SpotifyException: http status: 429, code:-1 - /v1/audio-features/?ids=1tkkrLY3upIwNfLs4ei8Bh:
 Max Retries, reason: too many 500 error responses
(app) /app #

Btw. I am working with several applications and CLIENT_IDs, I am sure I am not hitting rate limits (triple checked).

jkulak commented 2 years ago

@Peter-Schorn where you able to reproduce it? I am willing to help.

neilmenon commented 1 year ago

Hello @Peter-Schorn,

I've just had the same thing happen to me and see it reported here as well: https://github.com/home-assistant/core/issues/88547

spotify.playlist_upload_cover_image(created_playlist_id, base64_image)

gives

Exception: http status: 429, code:-1 - /v1/playlists/{id}/images: Max Retries, reason: too many 500 error responses

This was definitely not a rate-limit because it was the first call in several hours. Any ideas why it's being reported as 429?

Thanks!