search

ukwa / ukwa-heritrix

The UKWA Heritrix3 custom modules and Docker builder.
9 stars 7 forks source link

Cope with exceptions when scanning for metrics. #61

Open anjackson opened 3 years ago

anjackson commented 3 years ago 
dc_crawl_npld-dc-heritrix-worker.1.wd6kemo25c2v@ip-172-31-29-0    | Exception in thread "Thread-61" java.lang.NullPointerException
dc_crawl_npld-dc-heritrix-worker.1.wd6kemo25c2v@ip-172-31-29-0    |     at org.archive.crawler.frontier.WorkQueueFrontier.deepestUri(WorkQueueFrontier.java:1595)
dc_crawl_npld-dc-heritrix-worker.1.wd6kemo25c2v@ip-172-31-29-0    |     at uk.bl.wap.util.PrometheusMetricsExporterBean$CrawlJobMetricsThread.run(PrometheusMetricsExporterB
ean.java:277)

i.e. NPE at https://github.com/ukwa/ukwa-heritrix/blob/93420dcd76b0d51c864eeccf0c3c13a1464561ac/src/main/java/uk/bl/wap/util/PrometheusMetricsExporterBean.java#L277 killed the thead.

Bean is unnamed which makes getting hold of it difficult.

Can be restarted, using...