API hanging when making rapid requests

[gtpash]

It seems that the client hangs for me when I make rapid requests from the API. For example, I am trying to save every player ID to a file, but my code hangs after ~100 requests or so. About 10 minutes later it can request another 100, etc.

I figure this is an upstream issue, but was wondering if anyone had a workaround or had experienced something similar.

[ryant030408]

Yeah you should use a time.sleep for about two second in between each call, but there is a way to get every players info for a whole season in one call

let me look it up and ill post it

[ryant030408]

Try using player list in the player module, it should pull every player ever if you use current only set to 0

[rneu31]

Something like this:

from nba_py.player import PlayerList
# Get list of all players
player_list = PlayerList(season=season).info()

I also sleep between 1 and 1.5 seconds between each call. I created this function:

recorded_time = None
def sleep_if_needed():
    global recorded_time
    time_now = time.time()

    if recorded_time:
        time_delta = time_now - recorded_time
        rand_sleep = random.uniform(1.0, 1.5)
        if time_delta < rand_sleep:
            sleep = rand_sleep - time_delta
            print(f'Sleeping for {sleep} secs')
            time.sleep(sleep)

    recorded_time = time_now

And before making API calls I call that function

        sleep_if_needed()
        team_game_logs = TeamGameLogs(team_id).info()

Totally overkill. Figured it was worth sharing. Some sites are probably smart enough to check "are we getting tons of requests, all separated by EXACTLY 1 second?", but I made that up.

[gtpash]

Thanks for the replies. I'll probably get to this over the weekend and will update.

fyi: I'm doing this to stitch this data together with Basketball Reference pid's to get a dictionary between the two if anyone would be interested in having that. I can post that assuming I can finish it.

[gtpash]

Sorry for getting back so late. @ryant030408 your idea of putting the time.sleep call worked great.

The code for this project is on a university repo, but if anyone wants it just shoot me an email. I'm going to go ahead and close the issue.