search

lucidworks / auto-phrase-tokenfilter

Lucene Auto Phrase TokenFilter implementation
Other
59 stars 63 forks source link

using autphrase with solr 5 throws exception #12

Open MiladAlshomary opened 9 years ago

MiladAlshomary commented 9 years ago

Hello, I am using auto-phrase-tokenfilter with solr 5 and when am using the request handler am getting the following exception : java.lang.NoSuchMethodError: org.apache.lucene.analysis.core.WhitespaceTokenizer

Any Help ?

zniff commented 9 years ago

Hi it seems the constructor of WhitespaceTokenizer changed in Lucene 5.x

http://lucene.apache.org/core/5_2_1/analyzers-common/org/apache/lucene/analysis/core/WhitespaceTokenizer.html

So the code in question is most probably this one:

AutoPhrasingQParserPlugin.java: ... WhitespaceTokenizer wt = new WhitespaceTokenizer( new StringReader( input )); ...

must be changed to something like:

WhitespaceTokenizer wt = new WhitespaceTokenizer(); wt.setReader(new StringReader( input ));

cheers Erich

MiladAlshomary commented 9 years ago

Thanks, I updated the dependencies in ivy.xml and AutoPhrasingQParserPlugin.java and it worked fine. I forked your project and pushed the updates.

Eammi commented 8 years ago

Hello I was updating the AutoPhrasingQParserPlugin.java with the constructors for lucene 5.x as mentioned above and was trying to build with the solr 5.3.1 dependencies and I got an ant error like this:

test:
     [echo]
    [junit] WARNING: multiple versions of ant detected in path for junit
    [junit]          jar:file:/usr/share/ant/lib/ant.jar!/org/apache/tools/ant/Project.class
    [junit]      and jar:file:/usr/solr/auto/auto-phrase-tokenfilter/build-lib/ivy/lib/test/ant-1.8.2.jar!/org/apache/tools/ant/Project.class
    [junit] Running com.lucidworks.analysis.TestAutoPhrasingTokenFilter
    [junit] Testsuite: com.lucidworks.analysis.TestAutoPhrasingTokenFilter
    [junit] Tests run: 7, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 0.084 sec
    [junit] Tests run: 7, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 0.084 sec
    [junit]
    [junit] Testcase: testAutoPhraseEmitSingle took 0.057 sec
    [junit]     Caused an ERROR
    [junit] org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit] java.lang.NoSuchMethodError: org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit]     at com.lucidworks.analysis.TestAutoPhrasingTokenFilter.testAutoPhraseEmitSingle(Unknown Source)
    [junit]
    [junit] Testcase: testAutoPhrase took 0 sec
    [junit]     Caused an ERROR
    [junit] org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit] java.lang.NoSuchMethodError: org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit]     at com.lucidworks.analysis.TestAutoPhrasingTokenFilter.testAutoPhrase(Unknown Source)
    [junit]
    [junit] Testcase: testOverlappingAtBeginningEmitSingle took 0.001 sec
    [junit]     Caused an ERROR
    [junit] org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit] java.lang.NoSuchMethodError: org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit]     at com.lucidworks.analysis.TestAutoPhrasingTokenFilter.testOverlappingAtBeginningEmitSingle(Unknown Source)
    [junit]
    [junit] Testcase: testOverlappingAtBeginning took 0 sec
    [junit]     Caused an ERROR
    [junit] org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit] java.lang.NoSuchMethodError: org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit]     at com.lucidworks.analysis.TestAutoPhrasingTokenFilter.testOverlappingAtBeginning(Unknown Source)
    [junit]
    [junit] Testcase: testOverlappingAtEnd took 0 sec
    [junit]     Caused an ERROR
    [junit] org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit] java.lang.NoSuchMethodError: org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit]     at com.lucidworks.analysis.TestAutoPhrasingTokenFilter.testOverlappingAtEnd(Unknown Source)
    [junit]
    [junit] Testcase: testOverlappingAtEndEmitSingle took 0 sec
    [junit]     Caused an ERROR
    [junit] org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit] java.lang.NoSuchMethodError: org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit]     at com.lucidworks.analysis.TestAutoPhrasingTokenFilter.testOverlappingAtEndEmitSingle(Unknown Source)
    [junit]
    [junit] Testcase: testIncompletePhrase took 0 sec
    [junit]     Caused an ERROR
    [junit] org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit] java.lang.NoSuchMethodError: org.apache.lucene.analysis.core.WhitespaceTokenizer.<init>(Ljava/io/Reader;)V
    [junit]     at com.lucidworks.analysis.TestAutoPhrasingTokenFilter.testIncompletePhrase(Unknown Source)
    [junit]

BUILD FAILED
/usr/solr/auto/auto-phrase-tokenfilter/build.xml:75: Test com.lucidworks.analysis.TestAutoPhrasingTokenFilter failed
MiladAlshomary commented 8 years ago

To build the project against solr5 you can try this forked repository : https://github.com/MiladAlshomary/auto-phrase-tokenfilter It is updated and tested

Eammi commented 8 years ago

dear @MiladAlshomary thanks for your fast response. I have tried it with your forked repository and I am getting an error like this:

ivy-retrieve:
[ivy:retrieve] :: retrieving :: com.lucidworks.demo#autophrase-tokenfilter [sync]
[ivy:retrieve]  confs: [default, compile, test]
[ivy:retrieve]  210 artifacts copied, 0 already retrieved (174542kB/741ms)

set-classpaths:

compile:
    [javac] Compiling 3 source files to /usr/solr/autotest/auto-phrase-tokenfilter/build/java
    [javac] Note: /usr/solr/autotest/auto-phrase-tokenfilter/src/main/java/com/lucidworks/analysis/AutoPhrasingTokenFilter.java uses unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.

compile-test:
    [javac] Compiling 1 source file to /usr/solr/autotest/auto-phrase-tokenfilter/build/test
    [javac] /usr/solr/autotest/auto-phrase-tokenfilter/src/test/com/lucidworks/analysis/TestAutoPhrasingTokenFilter.java:21: error: no suitable constructor found for WhitespaceTokenizer(StringReader)
    [javac]     final WhitespaceTokenizer in = new WhitespaceTokenizer( reader );
    [javac]                                    ^
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer(AttributeFactory) is not applicable
    [javac]       (actual argument StringReader cannot be converted to AttributeFactory by method invocation conversion)
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer() is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /usr/solr/autotest/auto-phrase-tokenfilter/src/test/com/lucidworks/analysis/TestAutoPhrasingTokenFilter.java:64: error: no suitable constructor found for WhitespaceTokenizer(StringReader)
    [javac]     final WhitespaceTokenizer in = new WhitespaceTokenizer( reader );
    [javac]                                    ^
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer(AttributeFactory) is not applicable
    [javac]       (actual argument StringReader cannot be converted to AttributeFactory by method invocation conversion)
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer() is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /usr/solr/autotest/auto-phrase-tokenfilter/src/test/com/lucidworks/analysis/TestAutoPhrasingTokenFilter.java:118: error: no suitable constructor found for WhitespaceTokenizer(StringReader)
    [javac]     final WhitespaceTokenizer in = new WhitespaceTokenizer( reader );
    [javac]                                    ^
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer(AttributeFactory) is not applicable
    [javac]       (actual argument StringReader cannot be converted to AttributeFactory by method invocation conversion)
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer() is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /usr/solr/autotest/auto-phrase-tokenfilter/src/test/com/lucidworks/analysis/TestAutoPhrasingTokenFilter.java:140: error: no suitable constructor found for WhitespaceTokenizer(StringReader)
    [javac]     final WhitespaceTokenizer in = new WhitespaceTokenizer( reader );
    [javac]                                    ^
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer(AttributeFactory) is not applicable
    [javac]       (actual argument StringReader cannot be converted to AttributeFactory by method invocation conversion)
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer() is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /usr/solr/autotest/auto-phrase-tokenfilter/src/test/com/lucidworks/analysis/TestAutoPhrasingTokenFilter.java:170: error: no suitable constructor found for WhitespaceTokenizer(StringReader)
    [javac]     final WhitespaceTokenizer in = new WhitespaceTokenizer( reader );
    [javac]                                    ^
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer(AttributeFactory) is not applicable
    [javac]       (actual argument StringReader cannot be converted to AttributeFactory by method invocation conversion)
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer() is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /usr/solr/autotest/auto-phrase-tokenfilter/src/test/com/lucidworks/analysis/TestAutoPhrasingTokenFilter.java:202: error: no suitable constructor found for WhitespaceTokenizer(StringReader)
    [javac]     final WhitespaceTokenizer in = new WhitespaceTokenizer( reader );
    [javac]                                    ^
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer(AttributeFactory) is not applicable
    [javac]       (actual argument StringReader cannot be converted to AttributeFactory by method invocation conversion)
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer() is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] /usr/solr/autotest/auto-phrase-tokenfilter/src/test/com/lucidworks/analysis/TestAutoPhrasingTokenFilter.java:224: error: no suitable constructor found for WhitespaceTokenizer(StringReader)
    [javac]     final WhitespaceTokenizer in = new WhitespaceTokenizer( reader );
    [javac]                                    ^
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer(AttributeFactory) is not applicable
    [javac]       (actual argument StringReader cannot be converted to AttributeFactory by method invocation conversion)
    [javac]     constructor WhitespaceTokenizer.WhitespaceTokenizer() is not applicable
    [javac]       (actual and formal argument lists differ in length)
    [javac] 7 errors

BUILD FAILED
/usr/solr/autotest/auto-phrase-tokenfilter/build.xml:50: Compile failed; see the compiler error output for details.
MiladAlshomary commented 8 years ago

Hello Eammi, Please pull now and it must work. There was a problem in the test file.

Eammi commented 8 years ago

@MiladAlshomary thanks now it is working fine. I have created a lib folder inside the instanceDIR and have pasted the jar file inside. Is Solr 5 autoloading the plugin or do I need to define a lib directive? If yes could you provide a sample solrconfig.xml?

MiladAlshomary commented 8 years ago

@Eammi , you need to just drop the jar file inside the instance folder and use it as explained in the read me : https://github.com/LucidWorks/auto-phrase-tokenfilter/blob/master/README.md

maryjos commented 8 years ago

I'm using the forked version to build against Solr 5.4.0 and getting a failed dependency error, this location does not seem to exist any more: http://repo1.maven.org/maven2/org/restlet/

Looks like it might be available here: http://maven.restlet.com/org/restlet/jee/org.restlet/2.3.0/

But I'm not sure how I can even get it to go to that location, since this seems buried in the whole build process that is downloading the entire Solr server and all its dependencies. I updated the ivy.xml file to point to all the latest libraries but still no dice.

maryjos commented 8 years ago

Update: found the solution on StackOverflow here : http://stackoverflow.com/questions/30630227/java-ivy-maven-build-dependency-resolution-for-lucidworks-auto-phrase-tokenizer

Added code changes to the other fork and sent pull request.

varunvns commented 7 years ago

Hi Solr Experts,

I am a .Net Developer and currently working on Solr for a client requirement. I have changed taken Milad's GitHub repo code https://github.com/MiladAlshomary/auto-phrase-tokenfilter as I am using Solr 5.4.1

I am getting the following error:

Main error I feel is Caused by: java.lang.ClassCastException: class org.apache.lucene.analysis.tr.ApostropheFilterFactory

java.lang.ExceptionInInitializerError at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:274) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:543) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:628) at org.apache.solr.schema.FieldTypePluginLoader$3.create(FieldTypePluginLoader.java:382) at org.apache.solr.schema.FieldTypePluginLoader$3.create(FieldTypePluginLoader.java:376) at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:153) at org.apache.solr.schema.FieldTypePluginLoader.readAnalyzer(FieldTypePluginLoader.java:400) at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:105) at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:54) at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:153) at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:471) at org.apache.solr.schema.IndexSchema.(IndexSchema.java:160) at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:56) at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:70) at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:109) at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:80) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:744) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:466) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:457) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:232) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassCastException: class org.apache.lucene.analysis.tr.ApostropheFilterFactory at java.lang.Class.asSubclass(Class.java:3208) at org.apache.lucene.util.SPIClassIterator.next(SPIClassIterator.java:141) at org.apache.lucene.analysis.util.AnalysisSPILoader.reload(AnalysisSPILoader.java:79) at org.apache.lucene.analysis.util.AnalysisSPILoader.(AnalysisSPILoader.java:60) at org.apache.lucene.analysis.util.AnalysisSPILoader.(AnalysisSPILoader.java:49) at org.apache.lucene.analysis.util.TokenFilterFactory.(TokenFilterFactory.java:31)

I have added all the dependencies to my Solr lib. Still I am facing this error.

Can anyone please help?

Regards, Varun Shringarpure