Closed CarlosOliveira1988 closed 2 years ago
Based on the tests using the initial 'web-scraping-multiprocessing' branch, using the attached PORTFOLIO_TEMPLATE_PERFORMANCE.xlsx spreadsheet, we have the following results:
I've worked in a new approach in order to reduce the time to open new spreadsheets and also to launch the application. It seems the slowest tasks are related to web-scraping using 'BeautifulSoup' and 'requests' modules. I could not reduce their times for now. Maybe in the future I try other things.
Anyway, running the same mentioned spreadsheet with new code, I could reduce the time from 46 to 35 seconds, that means 24% less time.
Is your feature request related to a problem? Please describe.
The task of getting information from websites (through web-scrapping or YFinance API) takes too much time to be performed. This is really noticed when the application needs to work with 15 or more stocks.
Describe the solution you'd like
Apply the multiprocessing in the 'Renda Variável', 'Renda Fixa' and 'Tesouro Direto' tabs
Describe alternatives you've considered
Use the 'multiprocessing.Pool' library to perform the described activity. This link has useful information: https://www.analyticsvidhya.com/blog/2021/04/a-beginners-guide-to-multi-processing-in-python/
Additional context
Create a test spreadsheet with 20 opened positions per tab ('Renda Variável', 'Renda Fixa' and 'Tesouro Direto' tabs). Then, try to measure the current time and the improved time, in order to check the difference of processing time.