spotipy-dev / spotipy

A light weight Python library for the Spotify Web API
http://spotipy.readthedocs.org
MIT License
4.9k stars 954 forks source link

Get `HTTP 429` and error `Max Retries, reason: too many 500 error responses` when trying to fetch saved tracks. #1093

Open K4LCIFER opened 2 months ago

K4LCIFER commented 2 months ago

I have something similar to the following:

import spotipy
from spotipy.oauth2 import SpotifyOAuth

scope = "user-library-read user-read-recently-played"

sp = spotipy.Spotify(auth_manager=SpotifyOAuth(client_id="<client-id>", client_secret="<client-secret>", redirect_uri="http://localhost/", scope=scope))

print(sp.current_user_saved_tracks(market="US", limit=1))

But running this results in the following error

spotipy.exceptions.SpotifyException: http status: 429, code:-1 - /v1/me/tracks?limit=1&offset=0&market=US:
 Max Retries, reason: too many 500 error responses

This might be an error on Spotify's end, because, in the Spotify Web API docs, their API tester also errors out when attempting a similar request. Perhaps #1024 is related (I also have a lot of songs — approximately 40k)?

dieser-niko commented 2 months ago

You could try posting to the Spotify Community forum with the same question.

Your script seems to be fine, and you said yourself that it wasn't working in the documentation, which suggests that this is not a problem with spotipy.

K4LCIFER commented 2 months ago

You could try posting to the Spotify Community forum with the same question.

I have, but, unfortunately, the post never received a response. I have emailed Spotify's support back and forth, a few times, but I've yet to find a solution.

dieser-niko commented 2 months ago

That doesn't sound promising. I'll leave this issue open, even though this problem is not caused by spotipy. If there are any updates, please post them here.

I've also been experimenting a bit myself. First I created a playlist, because it's much easier to fill a playlist with a bunch of songs. Got up to 11,000, but no errors yet. Then I used my second account and around the 24,000 mark I got a code 500 with a "server error" message. So only partially reproducible, as I didn't get the same error you did. Not sure why I did this because it was clear to me that I couldn't fix anything about it, but yeah.. At least partially reproducible.

JosepVargasf commented 1 month ago

That response indicates that you have reached the rate limit of spotify API.

https://developer.spotify.com/documentation/web-api/concepts/rate-limits

dieser-niko commented 1 month ago

Uhm.. no. There may be an internal rate limit, but it has nothing to do with the rate limit the documentation is talking about. In this case it is not the application's fault.

I have already made sure that this is not a typical bug. Next time please read the comments.

dieser-niko commented 1 month ago

I was bored and poked around in the Web API documentation and found that the request goes through sometimes. So I guess a possible workaround would be to send multiple requests until you get a valid answer.

dieser-niko commented 1 month ago

@K4LCIFER Can you check if this is the same for you? You got a different error than I did, so I'm not 100% sure if this applies to you.

K4LCIFER commented 1 month ago

I tried it two ways: I ran the python script, which utilized spotipy, multiple times, and I tried the Spotify Web API documentation's API tester thing multiple times. In both cases I sent the request 30ish times, but they all responded with error 500.

dieser-niko commented 1 month ago

So it seems that the number of retries (default args) varies. First I only had to do 5, then 8, then 30, then 2, then 6.

Maybe you were unlucky, or maybe the extra 15k songs in your library are causing even more problems.