Update use of accumulators in parsing code

ScaleUnlimited / flink-crawler

Continuous scalable web crawler built on top of Flink and crawler-commons

Apache License 2.0

51 stars 18 forks source link

Closed kkrugler closed 6 years ago

kkrugler commented 6 years ago

Make sure parser's open() is called from ParseSiteMapFunction.
Push creation of CrawlerAccumulator into BasePageParser.
Call parser close() from ParseXXXFunction's close() method.
Add enum values for good/bad page parse results, and increment those in SimplePageParser