search

Norconex / committer-elasticsearch

Implementation of Norconex Committer for Elasticsearch.
https://opensource.norconex.com/committers/elasticsearch/
Apache License 2.0
11 stars 6 forks source link

Caused by: java.lang.IllegalArgumentException: Negative buffer size #34

Closed jmrichardson closed 5 years ago

jmrichardson commented 5 years ago

Hi,

I am getting these new error messages when committing to Elasticsearch. I am running a test on a small directory of ~200 files and none are being committed to ES. This is the error message from the log file:

com.norconex.committer.core.CommitterException: Could not commit JSON batch to Elasticsearch.
        at com.norconex.committer.elasticsearch.ElasticsearchCommitter.commitBatch(ElasticsearchCommitter.java:599)
        at com.norconex.committer.core.AbstractBatchCommitter.commitAndCleanBatch(AbstractBatchCommitter.java:179)
        at com.norconex.committer.core.AbstractBatchCommitter.cacheOperationAndCommitIfReady(AbstractBatchCommitter.java:208)
        at com.norconex.committer.core.AbstractBatchCommitter.commitAddition(AbstractBatchCommitter.java:143)
        at com.norconex.committer.core.AbstractFileQueueCommitter.commit(AbstractFileQueueCommitter.java:222)
        at com.norconex.committer.elasticsearch.ElasticsearchCommitter.commit(ElasticsearchCommitter.java:537)
        at com.norconex.collector.core.crawler.AbstractCrawler.execute(AbstractCrawler.java:274)
        at com.norconex.collector.core.crawler.AbstractCrawler.doExecute(AbstractCrawler.java:228)
        at com.norconex.collector.core.crawler.AbstractCrawler.startExecution(AbstractCrawler.java:184)
        at com.norconex.jef4.job.AbstractResumableJob.execute(AbstractResumableJob.java:49)
        at com.norconex.jef4.suite.JobSuite.runJob(JobSuite.java:355)
        at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:296)
        at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168)
        at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:132)
        at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95)
        at com.norconex.collector.fs.FilesystemCollector.main(FilesystemCollector.java:76)
Caused by: java.lang.IllegalArgumentException: Negative buffer size
        at java.io.StringWriter.<init>(StringWriter.java:67)
        at org.apache.commons.text.translate.CharSequenceTranslator.translate(CharSequenceTranslator.java:67)
        at org.apache.commons.text.StringEscapeUtils.escapeJson(StringEscapeUtils.java:585)
        at com.norconex.committer.elasticsearch.ElasticsearchCommitter.appendValue(ElasticsearchCommitter.java:727)
        at com.norconex.committer.elasticsearch.ElasticsearchCommitter.append(ElasticsearchCommitter.java:718)
        at com.norconex.committer.elasticsearch.ElasticsearchCommitter.append(ElasticsearchCommitter.java:697)
        at com.norconex.committer.elasticsearch.ElasticsearchCommitter.appendAddOperation(ElasticsearchCommitter.java:680)
        at com.norconex.committer.elasticsearch.ElasticsearchCommitter.commitBatch(ElasticsearchCommitter.java:577)
        ... 15 more

Thank you for your help, John

jmrichardson commented 5 years ago

Hey, this issue can be closed. We finally figured out there was an "artifact" or some sort of corrupt file in the commit directory that was causing the batch commit errors to elastic. We ended up removing the entire working directory and it fixed it. My understanding was the server crashed a few weeks ago due to storage space and this may have caused a corrupt file of some sort?

Thanks again for this great software!