Pipeline: Emit scraper special events & logs to some message broker

rivernews / review-scraper-java-development-environment

An environment to develop review scraper

0 stars 1 forks source link

Pipeline: Emit scraper special events & logs to some message broker #8

Closed rivernews closed 4 years ago

rivernews commented 4 years ago

To handle exceptions, such as:

Cannot unique locate company overview page
Necessary page access timed out, e.g. Company overview page, review page during any pagination, etc.
etc

rivernews commented 4 years ago

We are currently just using slack - solely for management purpose. Not planning on any log analytics. The slack bot is working well right now. Will close this ticket. Will be nice to implement in the future about scraper analytics.