Since the code challenge allows users to upload up to 100 keywords, the upload process will likely cause the request to time out. In addition, if there is any error during the scraping of one keyword, the other keywords will not be processed. For instance, the deployed application crashed:
Expected
The key challenge in this coding exercise is to implement background jobs. While it is not specified/said as-is, we want candidates to figure out the following processing of keywords:
Upload CSV
Validate the uploaded keywords (not empty, not more than 100 keywords)
Insert keywords in a database table (to ensure no data is lost, a flag attribute should be added to track the scraping status)
Issue
In the current implementation, keywords are processed one by one (in a loop) in the controller:
https://github.com/buigiaanfb1/extract-data/blob/10182a427a237113d64811701e57c7308d567509/app/controllers/search.controller.js#L28-L49
Since the code challenge allows users to upload up to 100 keywords, the upload process will likely cause the request to time out. In addition, if there is any error during the scraping of one keyword, the other keywords will not be processed. For instance, the deployed application crashed:
Expected
The key challenge in this coding exercise is to implement background jobs. While it is not specified/said as-is, we want candidates to figure out the following processing of keywords: