knox-academy / webscraping

0 stars 0 forks source link

Optimize the web scraper's performance and efficiency to ensure timely and accurate data retrieval. #36

Closed knox-academy closed 1 year ago

knox-academy commented 1 year ago

Mike McConnelly: Thanks, Dan. Here's the final issue description:

Issue Title: Optimize the web scraper's performance and efficiency to ensure timely and accurate data retrieval.

Issue Description: Our web scraper is currently experiencing slow and inaccurate data retrieval. We need to optimize its performance and efficiency to reduce the time it takes to retrieve a set amount of data by at least 50% and achieve a 99% success rate in retrieving the correct data. However, optimizing the scraper may require significant changes to the codebase, which could introduce new bugs or errors. Additionally, the scraper may need to be optimized for different types of websites or data sources, which could require additional development time. To determine if this issue is complete, we will measure the time it takes to retrieve a set amount of data and the success rate of retrieving the correct data before and after optimization.