search

apache / lucene

Apache Lucene open-source search software
https://lucene.apache.org/
Apache License 2.0
2.58k stars 1.01k forks source link

NPE in Japanese Analyzer [LUCENE-5244] #6308

Closed asfimport closed 10 years ago

asfimport commented 10 years ago

I've got a test case that shows an NPE with the Japanese analyzer.

It's all available in https://github.com/benson-basis/kuromoji-npe, and I explicitly grant a license to the Foundation.

If anyone would prefer that I attach a tarball here, just let me know.

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.basistech.testcase.JapaneseNpeTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.298 sec <<< FAILURE! - in com.basistech.testcase.JapaneseNpeTest
japaneseNpe(com.basistech.testcase.JapaneseNpeTest)  Time elapsed: 0.282 sec  <<< ERROR!
java.lang.NullPointerException: null
    at org.apache.lucene.analysis.util.RollingCharBuffer.get(RollingCharBuffer.java:86)
    at org.apache.lucene.analysis.ja.JapaneseTokenizer.parse(JapaneseTokenizer.java:618)
    at org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:468)
    at com.basistech.testcase.JapaneseNpeTest.japaneseNpe(JapaneseNpeTest.java:28)

Migrated from LUCENE-5244 by Benson Margulies (@bimargulies-google), resolved Sep 25 2013

asfimport commented 10 years ago

Robert Muir (@rmuir) (migrated from JIRA)

Your code does not consume correctly, the npe is intentional. In current Svn you get illegalstate...

asfimport commented 10 years ago

Christian Moen (@cmoen) (migrated from JIRA)

Hello Benson,

In your code on Github, try calling tokenStream.reset() before consumption.

asfimport commented 10 years ago

Benson Margulies (@bimargulies-google) (migrated from JIRA)

This was pilot error, I forgot to call reset().