set up github oauth to grab tons of data from their api

CulturedCheese / thesis-project

4 stars 4 forks source link

set up github oauth to grab tons of data from their api #195

Closed ClimbsRocks closed 9 years ago

ClimbsRocks commented 9 years ago

otherwise they rate limit you to 60 requests per hour

ClimbsRocks commented 9 years ago

did, and it was awesome. 23k api calls overnight, still staying below their rate limit. set the whole thing up (figured out the github api, figured out oauth, structured the queries to the api and storing into the db, and launched the staggered queries) in 1.5 hours before dinner.