Closed bdentino closed 9 years ago
superagent treats these responses as node errors, so x-ray thinks something went wrong with the request and errors out. I think this is bad behavior, especially if we're crawling multiple links. One down server will kill the whole scrape.
:heart_eyes:
matthewmueller
commented
9 years ago
superagent treats these responses as node errors, so x-ray thinks something went wrong with the request and errors out. I think this is bad behavior, especially if we're crawling multiple links. One down server will kill the whole scrape.