search

Norconex / jef

Job Execution Framework.
Apache License 2.0
4 stars 5 forks source link

NullPointerException with collector suite #10

Closed danizen closed 6 years ago

danizen commented 6 years ago

After a daily script stops the crawler as another process, I repeatedly see exceptions as follows:

FATAL [AbstractCrawler$ProcessReferencesRunnable] monitor_general_crawler: An error occured that could compromise the stability of the crawler. Stopping excution to avoid further issues...
java.lang.NullPointerException
        at com.norconex.jef4.job.group.AbstractJobGroup.groupProgressed(AbstractJobGroup.java:87)
        at com.norconex.jef4.suite.JobSuite$2.statusUpdated(JobSuite.java:372)
        at com.norconex.jef4.status.JobStatusUpdater.setProgress(JobStatusUpdater.java:50)
        at com.norconex.collector.core.crawler.AbstractCrawler.setProgress(AbstractCrawler.java:474)
        at com.norconex.collector.core.crawler.AbstractCrawler.processNextReference(AbstractCrawler.java:420)
        at com.norconex.collector.core.crawler.AbstractCrawler$ProcessReferencesRunnable.run(AbstractCrawler.java:812)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

This is simply a problem where the groupUpdater is set to null when stopping the suite, but groupProgressed does not check whether there is an updater.

essiembre commented 6 years ago

Merged and released (snapshot). Thank you for your patience.

danizen commented 6 years ago

No problem - if anything, this proved that using Artifactory on-site actually helps rather than hurts me, because I am able to depend on my patched version.