During scrapping I encounter couple pages that hangup during loading.
I'm doing scrapping on thousands of pages in parallel, and some of them stuck forever(at least 10+ min).
Timeout will solve this problem, ideally it would have user supplied value, but reasonable hardcode is also good.
During scrapping I encounter couple pages that hangup during loading. I'm doing scrapping on thousands of pages in parallel, and some of them stuck forever(at least 10+ min). Timeout will solve this problem, ideally it would have user supplied value, but reasonable hardcode is also good.