search

UCL-RITS / DashPykpi

Documentation
http://UCL-RITS.github.io/DashPykpi
MIT License
0 stars 0 forks source link

Speed up populating the database with async and await #18

Open benlaken opened 8 years ago

benlaken commented 8 years ago

@jenshnielsen mentioned that calling 

test = KpiStats(urls=urls)
test.work(verbose=False, debug=False, status=True)

can be sped up by the use of async methods. Investigate this to improve the code.

jenshnielsen commented 8 years ago

Note that I have no real knowlege about this but @benlaken mentioned that the bottleneck is getting requests back from the github api. This can probably be done in parallel but care needs to be taken to avoid flodding the github api and being banned. The relevant chapters of fluent python (orelly on ucl safari proquest ebooks) are useful.