Closed TobiGr closed 6 years ago
Why bother per_page
? Can't you fetch them all at once?
The limit per page is 100. I just make sure we reach it to have a solid base which we can use to do the calculation.
Well, my problem with this is that the requests are made sequentially.
I think I'd opt for some very basic web scraping, parsing the HTML site for that value instead.
We make just one request to get the number of contributors right now. So if you want, we could just fetch the contributors as done before my changes and append ?per_page=100&page=3
to the url. And if necessary, we can increment the page number by hand.
@TobiGr almost done with a very basic web scraping approach.
Check out #5.
This should fix the contributors count. @TheAssassin I am not familiar with Tornado, so I did the best I could. If you want, you can improve the performance - I couldn't because of my lack of knowledge and time.