search

TeamNewPipe / web-api

REST API providing external data feeds for the NewPipe website. Written as a micro service. In Python.
MIT License
22 stars 9 forks source link

Fix number of contributors #4

Closed TobiGr closed 6 years ago

TobiGr commented 6 years ago

This should fix the contributors count. @TheAssassin I am not familiar with Tornado, so I did the best I could. If you want, you can improve the performance - I couldn't because of my lack of knowledge and time.

TheAssassin commented 6 years ago

Why bother per_page? Can't you fetch them all at once?

TobiGr commented 6 years ago

The limit per page is 100. I just make sure we reach it to have a solid base which we can use to do the calculation.

TheAssassin commented 6 years ago

Well, my problem with this is that the requests are made sequentially.

I think I'd opt for some very basic web scraping, parsing the HTML site for that value instead.

TobiGr commented 6 years ago

We make just one request to get the number of contributors right now. So if you want, we could just fetch the contributors as done before my changes and append ?per_page=100&page=3 to the url. And if necessary, we can increment the page number by hand.

TheAssassin commented 6 years ago

@TobiGr almost done with a very basic web scraping approach.

TheAssassin commented 6 years ago

Check out #5.