search

shubhsherl / GSoC-Contribution-Leaderboard

This project is now moved to Node.js version and maintained https://github.com/lolimay/GSoC-Contribution-Leaderboard-Node.
https://gsoc.rocket.chat
MIT License
15 stars 16 forks source link

Improve refresh script to reduce runtime and eliminate occasional mis… #26

Closed kareemhamdy100 closed 5 years ago

kareemhamdy100 commented 5 years ago

…counting of Issue, PRs #20

the main problem is that you access database during fetch data from api

access database should done one time after you get all users data so I made that and try it for one repo "Rocket.chat" and the run time down from 7 minut to 3 minut in my machine

I hope I found away to reduce the updating database in less time my suggestion to update the showing data without store it and every one hour update the database that's maybe a good solution

shubhsherl commented 5 years ago

@kareemhamdy100 Can you run it at least for 4 or 5 repos. And while running check if all the API calls are 200 Success ? I guess the main problem is that some calls are getting response of 403 Rate limit.

Sing-Li commented 5 years ago

@kareemhamdy100 comment here ... let us know what improvement your latest update includes!

kareemhamdy100 commented 5 years ago

my code update all list in only 6 min

about 40 request per min so it never access 5000 request in hour

kareemhamdy100 commented 5 years ago

Improve :in first time it will get the all data in 8 ~11 min and after that will make refresh depend on the number of "open_issues" filed in each repo which refer to open issues +open PR if any of them changed the next request will update things related to the changed repo only

for the largest repo in rocketChat is Rocket.chat it's take about 2 min all others repos it take less than 1 min