The current codebase after adding web-crawler is missing some node dependencies. Resolve it so that it will work with npm i itself.
Implement proper error handling for web-crawler part.
Follow the response code convention like other APIs for this controller fully.
Rename the controller file so as to have Controller as suffix - to maintain uniformity (see other controller files).
Update the csvController.js accordingly after adding heading field. (Optionally, you can try to refactor the code so that any changes to contentModel doesn't require any changes to be made to this controller)
Migrate the route to scrape.js file instead of new file. (i.e, scrape/web-crawler can be the endpoint)
Return the number of Links scraped successfully along with failed links and any other such statistics that might be necessary as a part of successful response.
npm i
itself.csvController.js
accordingly after addingheading
field. (Optionally, you can try to refactor the code so that any changes to contentModel doesn't require any changes to be made to this controller)scrape.js
file instead of new file. (i.e,scrape/web-crawler
can be the endpoint)