Ability for scraper to push errors down the pipeline without throwing

[jzohrab]

Original issue https://github.com/covidatlas/coronadatascraper/issues/183, transferred here on Friday Mar 20, 2020 at 05:49 GMT

---

A couple ways this could work:

1. Scraper called with errors array as an argument that can be pushed to (feels weird man)
2. Scraper called with this = { errors: [] } (breaks a lot of scrapers)
3. Scraper called with region array as an argument you can push data to, can throw at any time (i.e. throw at the end of the scraper to indicate a non-fatal error)
4. Other ways?

[jzohrab]

(Transferred comment)

@qgolsteyn @ryanblock thoughts?

[jzohrab]

(Transferred comment)

What's an example of when this would be useful?

To me, idea 1 errors array isn't bad, it's a collector parameter. Have used such things in the past. Could even be a stream or a logger they push to.

• Other possibility: global logging, potentially. Scrapers could instantiate a ScraperError(this, err) and give it to a global logger routine. If scrapers know they can continue despite the error, great.