tokee / lucene-solr

High cardinality faceting (SOLR-5894)
http://tokee.github.io/lucene-solr/
7 stars 1 forks source link

getNonZeroBits in SplitRankZeroPlane sometimes fails #34

Open tokee opened 9 years ago

tokee commented 9 years ago

A test of sparse faceting using the nplanez counter gave multiple instances of the error below, each time with the same index 62266. The field had 3985012 unique references and since 62266*64 = 3985024, the error makes sense. Question is why the request was made at all.

INFO  - 2015-05-09 14:57:58.860; org.apache.solr.core.SolrCore; [collection1] webapp=/solr path=/select params={facet=true&efacet.collectors.compact=${COMPACT}&facet.mincount=1&facet.mincount=1&facet.limit=25&efacet.field=${FIELD}&facet.sparse.counting.threads.mindocs=10000&facet.sparse.counter=nplanez&ws=xml&efacet=false&facet.sparse=true&fl=id,url,host,domain,server,content_type_served,score,last_modified,content_type_tika,content_language&facet.sparse.log.extended=true&q=bimle&facet.sparse.termlookup=true&facet.sparse.packed=true&efacet.sparse=false&facet.counting.threads=1&facet.sparse.fraction=0.08&efacet.mincount=1&facet.field=links_domains&facet.sparse.skiprefinements=false&facet.sparse.mintags=100000} hits=2359 status=500 QTime=26 
ERROR - 2015-05-09 14:57:58.861; org.apache.solr.common.SolrException; null:org.apache.solr.common.SolrException: Exception during facet.field: links_domains
        at org.apache.solr.request.SimpleFacets$2.call(SimpleFacets.java:648)
        at org.apache.solr.request.SimpleFacets$2.call(SimpleFacets.java:633)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at org.apache.solr.request.SimpleFacets$1.execute(SimpleFacets.java:587)
        at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:658)
        at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:265)
        at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:89)
        at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
        at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:409)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1044)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IndexOutOfBoundsException: index 62266
        at java.util.concurrent.atomic.AtomicLongArray.checkedByteOffset(AtomicLongArray.java:64)
        at java.util.concurrent.atomic.AtomicLongArray.get(AtomicLongArray.java:111)
        at org.apache.lucene.util.packed.PackedOpportunistic.getBlock(PackedOpportunistic.java:87)
        at org.apache.lucene.util.packed.NPlaneMutable$SplitRankZeroPlane.getNonZeroBits(NPlaneMutable.java:940)
        at org.apache.lucene.util.packed.NPlaneMutable.getNonZeroBits(NPlaneMutable.java:143)
        at org.apache.solr.request.sparse.SparseCounterBitmap.iterate(SparseCounterBitmap.java:367)
        at org.apache.solr.request.sparse.SparseDocValuesFacets.getCounts(SparseDocValuesFacets.java:209)
        at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:418)
        at org.apache.solr.request.SimpleFacets$2.call(SimpleFacets.java:638)
        ... 26 more