[PATCH] NullPointerException when using nested SpanOrQuery in SpanNotQuery [LUCENE-352]

[asfimport]

Overview description: I'm using the span query classes in Lucene to generate higher scores for search results where the search terms are closer together. In certain situations I want to exclude terms from the span. When I attempt to exclude more than one term I get an error.

The example query I'm using is:

'brighton AND tourism' -pier -contents

I construct the query objects and the toString() version is:

spanNot(spanNear([contents:brighton contents:tourism], 10, false), spanOr([contents:pier, contents:road]))

Steps to reproduce:

1. Construct a SpanNearQuery (must have at least one term, but at least two makes more sense)
2. Construct a SpanOrQuery containing two or more terms
3. Construct a SpanNotQuery to include the first query object and exclude the second (SpanOrQuery)
4. Execute the search

Actual Results: A null pointer exception is thrown while generating the scores within the search.

Stack trace:
java.lang.NullPointerException
at
org.apache.lucene.search.spans.SpanOrQuery$1.doc(SpanOrQuery.java:174)
at
org.apache.lucene.search.spans.SpanNotQuery$1.next(SpanNotQuery.java:75)
at org.apache.lucene.search.spans.SpanScorer.next(SpanScorer.java:50)
at org.apache.lucene.search.Scorer.score(Scorer.java:37)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:92)
at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
at org.apache.lucene.search.Hits.<init>(Hits.java:43)
at org.apache.lucene.search.Searcher.search(Searcher.java:33)
at org.apache.lucene.search.Searcher.search(Searcher.java:27)
at
com.runtimecollective.search.LuceneSearch.search(LuceneSearch.java:362)

Expected Resuts: It executes the search and results where the first search terms (near query) are close together but without the second terms (or query) appearing.

---

Migrated from LUCENE-352 by Miles Barr, resolved May 18 2006 Environment:

Operating System: Linux
Platform: Other

Attachments: ASF.LICENSE.NOT.GRANTED--patch.txt, ASF.LICENSE.NOT.GRANTED--TestBasics.java.patch, ASF.LICENSE.NOT.GRANTED--TestBasicsPatch2.txt, LUCENE-352-consolidated-SpanNot-NPE.patch Linked issues:

• 1638

[asfimport]

Paul Elschot (migrated from JIRA)

The bug could be in SpanNotQuery.java, line 67, which sets moreExclude to true.

Could you try with:

    private boolean moreExclude = excludeSpans.next(); 

instead?

Regards, Paul Elschot

[asfimport]

Miles Barr (migrated from JIRA)

Created an attachment (id=14356) Change suggested by Paul Elschot

Changing moreExlude from 'true' to 'excludeSpans.next()' did the trick.

[asfimport]

Miles Barr (migrated from JIRA)

That should be 'moreExclude'.

[asfimport]

Paul Elschot (migrated from JIRA)

(In reply to comment #2) > Created an attachment (id=14356) [edit] > Change suggested by Paul Elschot >
> Changing moreExlude from 'true' to 'excludeSpans.next()' did the trick.

That's good news. Thanks for the patch.

Would it be possible for you to add a few test cases for this?

org.apache.lucune.search.spans.TestSpans in the src/test directory has some test examples.

[asfimport]

Miles Barr (migrated from JIRA)

Created an attachment (id=14357) Additional test cases for when using a SpanOrQuery as the exclude option

Here's a patch file for

test/src/org/apache/lucene/search/spans/TestBasics.java

I added two test cases:

1. When there's only one SpanQuery in the SpanOrQuery
2. When there's more than one SpanQuery in the SpanOrQuery

There's also a commented out test for when you have an empty SpanOrQuery but that currently throws an exception (complains that fields don't have the same name) when you try to run it.

I only used regular SpanTermQuery objects within the SpanOrQuery. I assume the regular SpanOrQuery tests verify the SpanOrQuery works correctly with the different span classes.

[asfimport]

Paul Elschot (migrated from JIRA)

Miles,

The commented test case in the patch to TestBasics passes an empty SpanOrQuery to a SpanNotQuery. That is probably not intended.

Regards, Paul Elschot

[asfimport]

Paul Elschot (migrated from JIRA)

(In reply to comment #6) > Miles,
>
> The commented test case in the patch to TestBasics passes an empty
> SpanOrQuery to a SpanNotQuery. That is probably not intended.

I just reread the previous post, and it is intended. However, I don't get the point of passing an empty SpanOrQuery.

>
> Regards,
> Paul Elschot
>

[asfimport]

Miles Barr (migrated from JIRA)

My reasoning was that a poorly written parser might construct it with an empty SpanOrQuery and it should probably produce the same results as just the include query. But it would be incorrect to do so, so you'd expect an error.

If you don't want to change the behaviour then just throw out the commented test case.

[asfimport]

Paul Elschot (migrated from JIRA)

Created an attachment (id=14558) Patch to TestBasics.java with the additional test cases except the discussed one.

I think this settles the discussion, Regards, Paul Elschot

[asfimport]

Paul Elschot (migrated from JIRA)

The patch of 24 Feb 2005 has not yet been applied. This bug might be reopened because of that, but that would be overdoing it.

The extra test code in the patch of 24 March 2005 seems ok to me.

Regards, Paul Elschot

[asfimport]

Paul Elschot (migrated from JIRA)

The fix in patch.txt is not in the trunk, and there was some recent discussion on what might be the same bug, so I'm reopening this.

[asfimport]

Chris M. Hostetter (@hossman) (migrated from JIRA)

consolidated patch containing...

• additional test methods in TestBasicsPatch2.txt
• additional test methods from #1638
• one line change to from patch.txt that makes these tests pass

[asfimport]

Chris M. Hostetter (@hossman) (migrated from JIRA)

I think this bug is actualy simple enough that I can honestly say I understand it ... so unless anyone obejcts i'll commit "LUCENE-352-consolidated-SpanNot-NPE.patch" in a few days.

[asfimport]

Yonik Seeley (@yonik) (migrated from JIRA)

+1 looks fine.

[asfimport]

Chris M. Hostetter (@hossman) (migrated from JIRA)

commited patch