Print warnings when a rate/request limit is reached

[dieser-niko]

I'm writing this sentence a fifth time. Does what the title says.

This is still a draft because I'm researching how to implement this so we can get the retry-after value right away. Currently it only prints a warning (whose text could be improved as well)

[dieser-niko]

After going a frame up, I found that Retry.increment was getting the full response object, including the header. Increment is used in several places (urllib3.connectionpool.HTTPConnectionPool.urlopen). The first one is on a redirect.

It might actually be important that we don't check for the Retry-After header until the second stage (when we actually check if it's a retry). I can't think of a better way than to override Retry.increment, where I'll check with Retry.is_retry whether it's a retry or not.

I feel that it can still fail, but the worst case would be a double entry in the logs, which would require a very niche setting for that to happen.

[dieser-niko]

If anyone wants to try this for themselves, just install

git+https://github.com/spotipy-dev/spotipy@retry-warning

I use this snippet to get a persistent rate limit after a few seconds:

import time

import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
import threading

sp = spotipy.Spotify(auth_manager=SpotifyClientCredentials(client_id="SPOTIPY_CLIENT_ID",
                                                           client_secret="SPOTIPY_CLIENT_SECRET"))

def do():
    sp.album_tracks("2Kh43m04B1UkVcpcRa1Zug")

while True:
    print(time.time())
    threading.Thread(target=do, daemon=True).start()
    time.sleep(0.01)

If you wait long enough, you'll probably get a request limit too, but nothing will change except the wait time.