spotDL / spotify-downloader

Download your Spotify playlists and songs along with album art and metadata (from YouTube if a match is found).
https://spotdl.readthedocs.io/en/latest/
MIT License
17.27k stars 1.59k forks source link

Max Treies reached: an error occured. ResponseError: too many 429 error responses #2209

Open Remheob opened 1 week ago

Remheob commented 1 week ago

System OS

Windows

Python Version

3.11 (CPython)

Install Source

pip / PyPi

Install version / commit hash

v4.2.8

Expected Behavior vs Actual Behavior

Wanted do download a playlist but it showd up some error. Tried it with an single song and it worked.

Steps to reproduce - Ensure to include actual links!

  1. spotdl download https://open.spotify.com/playlist/74ZTvAVerYgLngWeQ7EG9G?si=cc4a882280454baf&pt=d4edc815fae41afce5543421ae4e4976
  2. Get the error

I hope u can help me. At the end it says: The command ‘pt’ is either misspelled or could not be found.

Traceback

C:\Users\arneb>spotdl download https://open.spotify.com/playlist/74ZTvAVerYgLngWeQ7EG9G?si=cc4a882280454baf&pt=d4edc815fae41afce5543421ae4e4976
Processing query: https://open.spotify.com/playlist/74ZTvAVerYgLngWeQ7EG9G?si=cc4a882280454baf
Max Retries reached

An error occurred
ResponseError: too many 429 error responses

The above exception was the direct cause of the following exception:

┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\adapters.py:667 in send           │
│                                                                                                                     │
│   664 │   │   │   timeout = TimeoutSauce(connect=timeout, read=timeout)                                             │
│   665 │   │                                                                                                         │
│   666 │   │   try:                                                                                                  │
│ > 667 │   │   │   resp = conn.urlopen(                                                                              │
│   668 │   │   │   │   method=request.method,                                                                        │
│   669 │   │   │   │   url=url,                                                                                      │
│   670 │   │   │   │   body=request.body,                                                                            │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py:944 in urlopen   │
│                                                                                                                     │
│    941 │   │   │   response.drain_conn()                                                                            │
│    942 │   │   │   retries.sleep(response)                                                                          │
│    943 │   │   │   log.debug("Retry: %s", url)                                                                      │
│ >  944 │   │   │   return self.urlopen(                                                                             │
│    945 │   │   │   │   method,                                                                                      │
│    946 │   │   │   │   url,                                                                                         │
│    947 │   │   │   │   body,                                                                                        │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py:944 in urlopen   │
│                                                                                                                     │
│    941 │   │   │   response.drain_conn()                                                                            │
│    942 │   │   │   retries.sleep(response)                                                                          │
│    943 │   │   │   log.debug("Retry: %s", url)                                                                      │
│ >  944 │   │   │   return self.urlopen(                                                                             │
│    945 │   │   │   │   method,                                                                                      │
│    946 │   │   │   │   url,                                                                                         │
│    947 │   │   │   │   body,                                                                                        │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py:944 in urlopen   │
│                                                                                                                     │
│    941 │   │   │   response.drain_conn()                                                                            │
│    942 │   │   │   retries.sleep(response)                                                                          │
│    943 │   │   │   log.debug("Retry: %s", url)                                                                      │
│ >  944 │   │   │   return self.urlopen(                                                                             │
│    945 │   │   │   │   method,                                                                                      │
│    946 │   │   │   │   url,                                                                                         │
│    947 │   │   │   │   body,                                                                                        │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\connectionpool.py:934 in urlopen   │
│                                                                                                                     │
│    931 │   │   has_retry_after = bool(response.headers.get("Retry-After"))                                          │
│    932 │   │   if retries.is_retry(method, response.status, has_retry_after):                                       │
│    933 │   │   │   try:                                                                                             │
│ >  934 │   │   │   │   retries = retries.increment(method, url, response=response, _pool=self)                      │
│    935 │   │   │   except MaxRetryError:                                                                            │
│    936 │   │   │   │   if retries.raise_on_status:                                                                  │
│    937 │   │   │   │   │   response.drain_conn()                                                                    │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\urllib3\util\retry.py:519 in increment     │
│                                                                                                                     │
│   516 │   │                                                                                                         │
│   517 │   │   if new_retry.is_exhausted():                                                                          │
│   518 │   │   │   reason = error or ResponseError(cause)                                                            │
│ > 519 │   │   │   raise MaxRetryError(_pool, url, reason) from reason  # type:                                      │
│       ignore[arg-type]                                                                                              │
│   520 │   │                                                                                                         │
│   521 │   │   log.debug("Incremented Retry for (url='%s'): %r", url, new_retry)                                     │
│   522                                                                                                               │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
MaxRetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url:
/v1/playlists/74ZTvAVerYgLngWeQ7EG9G?additional_types=track (Caused by ResponseError('too many 429 error responses'))

During handling of the above exception, another exception occurred:

┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotipy\client.py:270 in _internal_call    │
│                                                                                                                     │
│    267 │   │   │   │   │    method, url, args.get("params"), headers, args.get('data'))                             │
│    268 │   │                                                                                                        │
│    269 │   │   try:                                                                                                 │
│ >  270 │   │   │   response = self._session.request(                                                                │
│    271 │   │   │   │   method, url, headers=headers, proxies=self.proxies,                                          │
│    272 │   │   │   │   timeout=self.requests_timeout, **args                                                        │
│    273 │   │   │   )                                                                                                │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py:589 in request        │
│                                                                                                                     │
│   586 │   │   │   "allow_redirects": allow_redirects,                                                               │
│   587 │   │   }                                                                                                     │
│   588 │   │   send_kwargs.update(settings)                                                                          │
│ > 589 │   │   resp = self.send(prep, **send_kwargs)                                                                 │
│   590 │   │                                                                                                         │
│   591 │   │   return resp                                                                                           │
│   592                                                                                                               │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\sessions.py:703 in send           │
│                                                                                                                     │
│   700 │   │   start = preferred_clock()                                                                             │
│   701 │   │                                                                                                         │
│   702 │   │   # Send the request                                                                                    │
│ > 703 │   │   r = adapter.send(request, **kwargs)                                                                   │
│   704 │   │                                                                                                         │
│   705 │   │   # Total elapsed time of the request (approximately)                                                   │
│   706 │   │   elapsed = preferred_clock() - start                                                                   │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\requests\adapters.py:691 in send           │
│                                                                                                                     │
│   688 │   │   │   │   │   raise ConnectTimeout(e, request=request)                                                  │
│   689 │   │   │                                                                                                     │
│   690 │   │   │   if isinstance(e.reason, ResponseError):                                                           │
│ > 691 │   │   │   │   raise RetryError(e, request=request)                                                          │
│   692 │   │   │                                                                                                     │
│   693 │   │   │   if isinstance(e.reason, _ProxyError):                                                             │
│   694 │   │   │   │   raise ProxyError(e, request=request)                                                          │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
RetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url:
/v1/playlists/74ZTvAVerYgLngWeQ7EG9G?additional_types=track (Caused by ResponseError('too many 429 error responses'))

During handling of the above exception, another exception occurred:

┌───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────┐
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotdl\console\entry_point.py:160 in       │
│ entry_point                                                                                                         │
│                                                                                                                     │
│   157 │   try:                                                                                                      │
│   158 │   │   # Pick the operation to perform                                                                       │
│   159 │   │   # based on the name and run it!                                                                       │
│ > 160 │   │   OPERATIONS[arguments.operation](                                                                      │
│   161 │   │   │   query=arguments.query,                                                                            │
│   162 │   │   │   downloader=downloader,                                                                            │
│   163 │   │   )                                                                                                     │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotdl\console\download.py:25 in download  │
│                                                                                                                     │
│   22 │   """                                                                                                        │
│   23 │                                                                                                              │
│   24 │   # Parse the query                                                                                          │
│ > 25 │   songs = get_simple_songs(                                                                                  │
│   26 │   │   query,                                                                                                 │
│   27 │   │   use_ytm_data=downloader.settings["ytm_data"],                                                          │
│   28 │   │   playlist_numbering=downloader.settings["playlist_numbering"],                                          │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotdl\utils\search.py:260 in              │
│ get_simple_songs                                                                                                    │
│                                                                                                                     │
│   257 │   │   │   )                                                                                                 │
│   258 │   │   │   songs.extend(full_lists)                                                                          │
│   259 │   │   elif "open.spotify.com" in request and "playlist" in request:                                         │
│ > 260 │   │   │   lists.append(Playlist.from_url(request, fetch_songs=False))                                       │
│   261 │   │   elif "open.spotify.com" in request and "album" in request:                                            │
│   262 │   │   │   lists.append(Album.from_url(request, fetch_songs=False))                                          │
│   263 │   │   elif "open.spotify.com" in request and "artist" in request:                                           │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotdl\types\song.py:306 in from_url       │
│                                                                                                                     │
│   303 │   │   - The SongList object.                                                                                │
│   304 │   │   """                                                                                                   │
│   305 │   │                                                                                                         │
│ > 306 │   │   metadata, songs = cls.get_metadata(url)                                                               │
│   307 │   │   urls = [song.url for song in songs]                                                                   │
│   308 │   │                                                                                                         │
│   309 │   │   if fetch_songs:                                                                                       │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotdl\types\playlist.py:48 in             │
│ get_metadata                                                                                                        │
│                                                                                                                     │
│    45 │   │                                                                                                         │
│    46 │   │   spotify_client = SpotifyClient()                                                                      │
│    47 │   │                                                                                                         │
│ >  48 │   │   playlist = spotify_client.playlist(url)                                                               │
│    49 │   │   if playlist is None:                                                                                  │
│    50 │   │   │   raise PlaylistError("Invalid playlist URL.")                                                      │
│    51                                                                                                               │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotipy\client.py:666 in playlist          │
│                                                                                                                     │
│    663 │   │   │   │   │   │   │   │   │    valid types are: track and episode                                      │
│    664 │   │   """                                                                                                  │
│    665 │   │   plid = self._get_id("playlist", playlist_id)                                                         │
│ >  666 │   │   return self._get(                                                                                    │
│    667 │   │   │   f"playlists/{plid}",                                                                             │
│    668 │   │   │   fields=fields,                                                                                   │
│    669 │   │   │   market=market,                                                                                   │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotdl\utils\spotify.py:195 in _get        │
│                                                                                                                     │
│   192 │   │   retries = self.max_retries  # type: ignore # pylint: disable=E1101                                    │
│   193 │   │   while response is None:                                                                               │
│   194 │   │   │   try:                                                                                              │
│ > 195 │   │   │   │   response = self._internal_call("GET", url, payload, kwargs)                                   │
│   196 │   │   │   except (requests.exceptions.Timeout, requests.ConnectionError) as exc:                            │
│   197 │   │   │   │   retries -= 1                                                                                  │
│   198 │   │   │   │   if retries <= 0:                                                                              │
│                                                                                                                     │
│ C:\Users\arneb\AppData\Local\Programs\Python\Python312\Lib\site-packages\spotipy\client.py:311 in _internal_call    │
│                                                                                                                     │
│    308 │   │   │   │   reason = retry_error.args[0].reason                                                          │
│    309 │   │   │   except (IndexError, AttributeError):                                                             │
│    310 │   │   │   │   reason = None                                                                                │
│ >  311 │   │   │   raise SpotifyException(                                                                          │
│    312 │   │   │   │   429,                                                                                         │
│    313 │   │   │   │   -1,                                                                                          │
│    314 │   │   │   │   f"{request.path_url}:\n Max Retries",                                                        │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
SpotifyException: http status: 429, code:-1 - /v1/playlists/74ZTvAVerYgLngWeQ7EG9G?additional_types=track:
 Max Retries, reason: too many 429 error responses
Der Befehl "pt" ist entweder falsch geschrieben oder
konnte nicht gefunden werden. -> It means "The command ‘pt’ is either misspelled or
could not be found."

Other details

No response

sledgemhammer commented 6 days ago

Encountered the same problem on any playlist how many tracks does not matter, some rate limit is hit at the first song when using spotdl without spotify credentials. (Same with spotipy)

Problem occurs sometimes also with a single song ( no playlist), after waiting some time and retry : the single song suddenly starts downloading without error.

Found a solution for anyone reading : Signup with spotify developer account. (redirect url use by me : https://development.spotify.com, the rest is obvious) Copy your Client_ID and Secret_ID to notepad Use those with spotdl as parameters ( see help on how to use with -- clientID and --clientSecret).

Your problem will be solved this way and you can download playlists again without 429 error. I think this error is triggered on the spotify platform and not on spotdl

On spotipy (underlaying for spotdl) they write about new rate limit mechanisam. I think its more strict on anonymous requests.

mirlansmind commented 6 days ago

The error message comes when it can not find the equivalent link on youtube I am guessing. I wrote a little bash file that goes thru track links in a .vim file (manually copy pasted from the album) and downloads the ones it can find and skips the ones it can't.

  1. Make a download.sh file and copy this code into it

!/bin/bash

if [ $# -eq 0 ]; then echo "Usage: $0 " exit 1 fi

while IFS= read -r link do if [[ ! -z "$link" ]]; then echo "Downloading: $link"

Download the link and wait for it to finish

    spotdl download "$link"
    if [ $? -eq 0 ]; then
        echo "Download completed: $link"
    else
        echo "Error downloading: $link"
        continue 
    fi
fi
  1. On the terminal in the working directory run "chmod +x download.sh"
  2. Then make a file links.vim and copy paste track links from the album one by one in new rows (yes this is the manual part I could not automate"
  3. Run ./download.sh links.vim
sledgemhammer commented 5 days ago

The error message comes when it can not find the equivalent link on youtube I am guessing. I wrote a little bash file that goes thru track links in a .vim file (manually copy pasted from the album) and downloads the ones it can find and skips the ones it can't.

I dont agree, and tried what you said to see if it cannot find. True in some cases but not all cases. Proof comes with the working download if you supply clientID and cliendSecret into the request. ( when it was previous not working it is working this way, if a file not found it will download the rest ánd gives a message for the not found file)

I was able to download a playlist > 100 tracks this way. I am sure its on the spotify side initiated with 3 missing tracks