We need a function to make requests to extract web pages HTML code. This function needs to:
Abstract the library used to make the requests: it needs to be a wrapper. This is needed because I will probably switch to the native implementation of fetch in Node.js, once it's released in the LTS version.
Implement a retry mechanism with a variable number of retries: if we make a lot of requests at the same time, we could be getting 5xx errors.
The retry mechanism has to have an exponential wait time between retries: to leave enough room for the temporal fail to disappear.
Also, as this is going to be the first function of the project, we also need to set up tests configuration.
We need a function to make requests to extract web pages HTML code. This function needs to:
fetchin Node.js, once it's released in the LTS version.Also, as this is going to be the first function of the project, we also need to set up tests configuration.