search

Stratio / cassandra-lucene-index

Lucene based secondary indexes for Cassandra
Apache License 2.0
600 stars 170 forks source link

Lucene index creation fails in unit test cases for cassandra 2.2.6 #239

Open shasidhareranti opened 7 years ago

shasidhareranti commented 7 years ago 
 [testng] 22:07:50.731 [SharedPool-Worker-2] DEBUG c.s.c.lucene.service.LuceneIndex - actional_dashboard.reportconfig.lucene update document Document docValuesType=SORTED_SET indexed,tokenized,omitNorms,indexOptions=DOCS docValuesType=SORTED_SET indexed,tokenized,omitNorms,indexOptions=DOCS docValuesType=SORTED_SET indexed,tokenized,omitNorms,indexOptions=DOCS docValuesType=SORTED_SET indexed,tokenized,omitNorms,indexOptions=DOCS,numericType=LONG,numericPrecisionStep=16,docValuesType=NUMERIC<_token:3173195065897216566> stored,indexed,omitNorms,indexOptions=DOCS,docValuesType=SORTED<_partition_key:[66 7 b8 52 c8 e0 40 44 88 e7 b4 13 12 7a ce 56]> stored,indexed,omitNorms,indexOptions=DOCS,docValuesType=SORTED<_primary_key:[0 10 61 63 30 39 37 34 31 39 61 31 64 33 36 61 33 36 0 0 10 66 7 b8 52 c8 e0 40 44 88 e7 b4 13 12 7a ce 56 0 0 19 0 13 72 65 70 6f 72 74 5f 67 72 6f 75 70 5f 63 6f 6e 66 69 67 0 0 0 0 0]>> with term _primary_key:[0 10 61 63 30 39 37 34 31 39 61 31 64 33 36 61 33 36 0 0 10 66 7 b8 52 c8 e0 40 44 88 e7 b4 13 12 7a ce 56 0 0 19 0 13 72 65 70 6f 72 74 5f 67 72 6f 75 70 5f 63 6f 6e 66 69 67 0 0 0 0 0]
   [testng] 22:07:50.871 [SharedPool-Worker-2] WARN  o.a.c.c.AbstractLocalAwareExecutorService - Uncaught exception on thread Thread[SharedPool-Worker-2,10,main]: {}
   [testng] java.lang.AssertionError: TokenStream implementation classes or at least their incrementToken() implementation must be final
   [testng]     at org.apache.lucene.analysis.TokenStream.assertFinal(TokenStream.java:126) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.analysis.TokenStream.(TokenStream.java:99) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.analysis.Tokenizer.(Tokenizer.java:45) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.analysis.core.KeywordTokenizer.(KeywordTokenizer.java:43) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.analysis.core.KeywordTokenizer.(KeywordTokenizer.java:40) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.analysis.core.KeywordAnalyzer.createComponents(KeywordAnalyzer.java:32) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.analysis.AnalyzerWrapper.createComponents(AnalyzerWrapper.java:101) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.analysis.AnalyzerWrapper.createComponents(AnalyzerWrapper.java:101) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:176) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.document.Field.tokenStream(Field.java:562) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:628) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:365) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:321) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:234) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:450) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1477) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at com.stratio.cassandra.lucene.service.LuceneIndex.upsert(LuceneIndex.java:131) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at com.stratio.cassandra.lucene.service.LuceneIndex.upsert(LuceneIndex.java:145) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at com.stratio.cassandra.lucene.service.RowServiceWide.doIndex(RowServiceWide.java:98) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at com.stratio.cassandra.lucene.service.RowService.index(RowService.java:139) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at com.stratio.cassandra.lucene.Index.index(Index.java:118) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]
   [testng]     at org.apache.cassandra.db.index.SecondaryIndexManager$StandardUpdater.updateRowLevelIndexes(SecondaryIndexManager.java:861) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at org.apache.cassandra.db.AtomicBTreeColumns.addAllWithSizeDelta(AtomicBTreeColumns.java:238) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at org.apache.cassandra.db.Memtable.put(Memtable.java:230) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1263) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:406) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:366) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at org.apache.cassandra.db.Mutation.apply(Mutation.java:214) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at org.apache.cassandra.service.StorageProxy$8.runMayThrow(StorageProxy.java:1110) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2272) ~[cassandra-all-2.2.6.jar:2.2.6]
   [testng]     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_91]
   [testng]     at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[cassandra-all-2.2.6.jar:2.2.6]
shasidhareranti commented 7 years ago

@adelapena Is this normal behaviour while updating indexes in unit test cases?

ealonsodb commented 7 years ago

Hi @shasidhareranti

If you run mvn clean test Does the unit test complete successfully?

It should show something similar to this:

Results :

Tests run: 1283, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Cassandra Lucene index ............................ SUCCESS [0.532s]
[INFO] Cassandra Lucene Index builder .................... SUCCESS [2.593s]
[INFO] Cassandra Lucene Index plugin ..................... SUCCESS [17.052s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 20.417s
[INFO] Finished at: Mon Nov 21 10:36:16 CET 2016
[INFO] Final Memory: 51M/510M
[INFO] ------------------------------------------------------------------------

You are using version 2.2.6.2, aren't you?

shasidhareranti commented 7 years ago

Hi What i mean is, when i try to create index in our application unit tests the index updation fails. In our application we start cassandra server, create tables with required UDTs , indexes and try to add some data.

By adding new rows index updation is triggered in background, which is failing with the error that i have posted above.

ealonsodb commented 7 years ago

Hi @shasidhareranti : Can you please show us the exact unit test in CQL?

adelapena commented 7 years ago

The problem seems to point to a wrong/incompatible implementation of a Lucene analyzer being called during the tests.

shasidhareranti commented 7 years ago

We are using cassandra-lucene-index-plugin-2.2.6.2.jar. Yes as part of unit tests we are running embedded cassandra server. There are no other conflicting dependency as you can observe in the stack trace as below.

   [testng] java.lang.AssertionError: TokenStream implementation classes or at least their incrementToken() implementation must be final
   [testng]     at org.apache.lucene.analysis.TokenStream.assertFinal(TokenStream.java:126) ~[cassandra-lucene-index-plugin-2.2.6.2.jar:na]

We are using the following index in our tables

{"default_analyzer":"english","fields":{"type":{"type":"string","case_sensitive":false},"title":{"type":"string","case_sensitive":false},"labels":{"type":"string","case_sensitive":false}}}