search

tjake / Solandra

Solandra = Solr + Cassandra
Apache License 2.0
882 stars 150 forks source link

NullPointerExceptions when using dismax #50

Closed karussell closed 13 years ago

karussell commented 13 years ago

When using the demo:

Now sometimes the exception does not occur after data feeding. But the initial query always failed in my tests.

When I'm querying while feeding sometimes other exception (randomly?) occured E.g. http://localhost:8983/solandra/reuters/select?q=text&qt=dismax =>

    java.lang.NullPointerException
at lucandra.LucandraTermEnum.skipTo(LucandraTermEnum.java:87)
at lucandra.LucandraTermDocs.seek(LucandraTermDocs.java:115)
at org.apache.lucene.index.IndexReader.termDocs(IndexReader.java:1103)
at lucandra.IndexReader.termDocs(IndexReader.java:406)
at org.apache.solr.search.SolrIndexReader.termDocs(SolrIndexReader.java:320)
at org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:67)
at org.apache.lucene.search.DisjunctionMaxQuery$DisjunctionMaxWeight.scorer(DisjunctionMaxQuery.java:144)
at org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:297)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:250)
at org.apache.lucene.search.Searcher.search(Searcher.java:171)
at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at solandra.SolandraDispatchFilter.execute(SolandraDispatchFilter.java:169)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at solandra.SolandraDispatchFilter.doFilter(SolandraDispatchFilter.java:133)

and after refresh this one happened:

java.lang.NullPointerException at lucandra.LucandraTermEnum.docFreq(LucandraTermEnum.java:99) at lucandra.IndexReader.docFreq(IndexReader.java:177) at org.apache.solr.search.SolrIndexReader.docFreq(SolrIndexReader.java:308) at org.apache.lucene.search.IndexSearcher.docFreq(IndexSearcher.java:147) at org.apache.lucene.search.Similarity.idfExplain(Similarity.java:765) at org.apache.lucene.search.TermQuery$TermWeight.(TermQuery.java:46) at org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:146) at org.apache.lucene.search.DisjunctionMaxQuery$DisjunctionMaxWeight.(DisjunctionMaxQuery.java:106) at org.apache.lucene.search.DisjunctionMaxQuery.createWeight(DisjunctionMaxQuery.java:177) at org.apache.lucene.search.BooleanQuery$BooleanWeight.(BooleanQuery.java:184) at org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:415) at org.apache.lucene.search.Query.weight(Query.java:99) at org.apache.lucene.search.Searcher.createWeight(Searcher.java:230) at org.apache.lucene.search.Searcher.search(Searcher.java:171) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) at solandra.SolandraDispatchFilter.execute(SolandraDispatchFilter.java:169) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at solandra.SolandraDispatchFilter.doFilter(SolandraDispatchFilter.java:133)

tjake commented 13 years ago

Thanks, I'll be pushing a bunch of internal changes soon and will address this

tjake commented 13 years ago

I've pushed the changes I mentioned, please take a look.

No longer getting exceptions but the dismax queries seem to only return results after a similar query on standard search handler. I must be doing something wrong.

karussell commented 13 years ago

Could it be that simply the following is missing from the dismax handler section?

 <arr name="components">
  <str>query</str>
  <str>solandraComponent</str>
  <str>facet</str>
  <str>mlt</str>
  <str>highlight</str>
  <str>stats</str>
  <str>debug</str>
 </arr>

Without that I didn't got results. With that all seems to be ok.

I'll report in tomorrow (or in the next days ;)) if my usecase for dismax still fails with that addition.

karussell commented 13 years ago

Ok, nice. Feeding and querying from jetwick works now (with the above change) :-)

So, thanks for the changes!

tjake commented 13 years ago

Cool is it live? what's the url?

karussell commented 13 years ago

sadly not yet, due too some other problems (see new issues). Also sorting seems to be slow. Or is only sort by latest date supported?

tjake commented 13 years ago

How many many documents in the indexes?

Are you running this on a single node? Solandra won't be faster than Solr on a single node keep in mind. Also you have shards.at.once=2 which doesn't make sense if you have 1 node. It should roughly be the same as the number of nodes in your cluster.

karussell commented 13 years ago

How many many documents in the indexes?

there were only 5K I think

Are you running this on a single node?

for this test, yes.

Solandra won't be faster than Solr on a single node keep in mind.

Also for large indices >10GB?

Also you have shards.at.once=2 which doesn't make sense if you have 1 node.

ok

karussell commented 13 years ago

Also you have shards.at.once=2 which doesn't make sense if you have 1 node.

also if the node has more than one core?

tjake commented 13 years ago

it will help with write tps but it will increase the amount of memory needed :)

karussell commented 13 years ago

ah, now I understand. so "at.once" is important here ... sorry

tjake commented 13 years ago

How many documents in your index?

karussell commented 13 years ago

ok, but this issue here is fixed. I'll close, ok?

karussell commented 13 years ago

How many documents in your index?

there were only 5K I think