Open HarryC145 opened 8 years ago
If a site is having technical issues, so say has gone down, the site should be skipped in that grab so time is not wasted. This could be implemented by "If x out of x URLs are not responding, then stop"
If a site is having technical issues, so say has gone down, the site should be skipped in that grab so time is not wasted. This could be implemented by "If x out of x URLs are not responding, then stop"