I've (accidentally) added a very large attribute (containing the lshw output) to some of my systems. Now when collins tries to update solr it runs into this error:
2016-09-27 16:21:52.406254500 [error] o.a.s.c.SolrCore - org.apache.solr.common.SolrException: Exception writing document id ASSET_2 to the index; possible analysis error.
2016-09-27 16:21:52.406279500 at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:167)
2016-09-27 16:21:52.406281500 at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
2016-09-27 16:21:52.406282500 at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
2016-09-27 16:21:52.406284500 at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:955)
2016-09-27 16:21:52.406285500 at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1110)
2016-09-27 16:21:52.406286500 at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:706)
2016-09-27 16:21:52.406287500 at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:250)
2016-09-27 16:21:52.406289500 at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
2016-09-27 16:21:52.406290500 at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
2016-09-27 16:21:52.406291500 at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
2016-09-27 16:21:52.406293500 at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
2016-09-27 16:21:52.406294500 at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
2016-09-27 16:21:52.406295500 at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:179)
2016-09-27 16:21:52.406296500 at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
2016-09-27 16:21:52.406298500 at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:107)
2016-09-27 16:21:52.406299500 at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:72)
2016-09-27 16:21:52.406300500 at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:86)
2016-09-27 16:21:52.406301500 at collins.solr.SolrHelper$$anonfun$updateItems$1.apply(SolrHelper.scala:120)
2016-09-27 16:21:52.406302500 at collins.solr.SolrHelper$$anonfun$updateItems$1.apply(SolrHelper.scala:115)
2016-09-27 16:21:52.406304500 at scala.Option.map(Option.scala:146)
2016-09-27 16:21:52.406305500 at collins.solr.SolrHelper$.updateItems(SolrHelper.scala:115)
2016-09-27 16:21:52.406306500 at collins.solr.SolrHelper$.updateAssets(SolrHelper.scala:137)
2016-09-27 16:21:52.406307500 at collins.solr.SolrHelper$$anonfun$populate$1$$anonfun$apply$2.apply(SolrHelper.scala:98)
2016-09-27 16:21:52.406308500 at collins.solr.SolrHelper$$anonfun$populate$1$$anonfun$apply$2.apply(SolrHelper.scala:91)
2016-09-27 16:21:52.406310500 at scala.Option.map(Option.scala:146)
2016-09-27 16:21:52.406311500 at collins.solr.SolrHelper$$anonfun$populate$1.apply(SolrHelper.scala:91)
2016-09-27 16:21:52.406312500 at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
2016-09-27 16:21:52.406313500 at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
2016-09-27 16:21:52.406314500 at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
2016-09-27 16:21:52.406316500 at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
2016-09-27 16:21:52.406317500 at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
2016-09-27 16:21:52.406318500 at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
2016-09-27 16:21:52.406319500 at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
2016-09-27 16:21:52.406320500 at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
2016-09-27 16:21:52.406321500 Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="LSHW_meta_s" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[1, 10, 62, 116, 115, 105, 108, 47, 60, 10, 62, 101, 100, 111, 110, 47, 60, 10, 62, 101, 100, 111, 110, 47, 60, 32, 32, 10, 62, 115]...', original message: bytes can be at most 32766 in length; got 128174
2016-09-27 16:21:52.406323500 at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:671)
2016-09-27 16:21:52.406324500 at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:344)
2016-09-27 16:21:52.406325500 at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:300)
2016-09-27 16:21:52.406327500 at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234)
2016-09-27 16:21:52.406328500 at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450)
2016-09-27 16:21:52.406329500 at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1475)
2016-09-27 16:21:52.406330500 at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:239)
2016-09-27 16:21:52.406331500 at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:163)
2016-09-27 16:21:52.406332500 ... 33 more
2016-09-27 16:21:52.406333500 Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 128174
2016-09-27 16:21:52.406335500 at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)
2016-09-27 16:21:52.406336500 at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:150)
2016-09-27 16:21:52.406337500 at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:661)
2016-09-27 16:21:52.406338500 ... 40 more
I'm running a pretty recent built from master (built August 10).
Since I didn't run into this earlier, I'm not completely sure if the attribute is simply too long or whether there is some regression. If it's simply too long, the API should prevent me from adding those.
Hi,
I've (accidentally) added a very large attribute (containing the lshw output) to some of my systems. Now when collins tries to update solr it runs into this error:
I'm running a pretty recent built from master (built August 10).
Since I didn't run into this earlier, I'm not completely sure if the attribute is simply too long or whether there is some regression. If it's simply too long, the API should prevent me from adding those.