search

Norconex / jef-monitor

Web-based application for monitoring jobs progress (created with JEF).
4 stars 3 forks source link

JEF Monitor causes crawl failures on Windows #9

Open essiembre opened 5 years ago

essiembre commented 5 years ago

From collector-http created by rustyx : Norconex/collector-http#519

Due to file locking, a running JEF Monitor causes intermittent crawler start failures with an exception like this:

java.io.IOException: Could not move "D:\work\logs\latest\logs\test1.log" to "D:\work\logs\backup\2018\09\05\logs\201809052154420193__test1.log".
        at com.norconex.commons.lang.file.FileUtil.moveFile(FileUtil.java:179)
        at com.norconex.jef4.log.FileLogManager.backup(FileLogManager.java:186)
        at com.norconex.jef4.suite.JobSuite.backupSuite(JobSuite.java:543)
        at com.norconex.jef4.suite.JobSuite.initialize(JobSuite.java:473)
        at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:277)
        at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168)
        at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:131)
        at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95)
        at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74)

Windows error message is: The process cannot access the file because it is being used by another process

truezjz commented 5 years ago

Hi Pascal, for this issue, instead of using file logs, using RDB to log will be a solution, just like ManifoldCF did.

essiembre commented 5 years ago

Something to consider for sure. Right now, it is possible to write your own LogManager to store logs however you want. Probably not the easiest. The next major version will use SLF4J and you will be able to plug your own logging implementation.