Performance Tuning for get_new_results

LuminosoInsight / sales-engineering-code

Code for sales engineering, particularly for code that will be given to customers

MIT License

0 stars 0 forks source link

Performance Tuning for get_new_results #69

Closed shiumachi closed 5 years ago

shiumachi commented 5 years ago

I introduced multi-threading with concurrent.futures to improve the performance in get_new_results(). With this PR, get_new_results() will be 84% faster than the current version against a 2,000 documents project.

master: 40.925 sec
this PR: 22.142 sec

In this code, LuminosoClient object is created in each thread, because requests.Session is not 100% threadsafe.