testing with embedded cassandra: Failed to validate custom indexer options

[Merkaban]

Hi, we start using an embedded cassandra for integration testing. In the application itself everything works fine, but in testing I get the following error:

Caused by: com.datastax.driver.core.exceptions.InvalidConfigurationInQueryException: Failed to validate custom indexer options: {schema={       default_analyzer : "english",       fields : {          customerid      : {type : "string"},            businessstate   : {type : "string"},            gatewayid       : {type : "string"},            start           : {type : "date", pattern: "yyyy-MM-dd HH:mm:ss.SSS"},          end             : {type : "date", pattern: "yyyy-MM-dd HH:mm:ss.SSS"},          duration        : {type : "date_range", from : "start", to : "end", pattern: "yyyy-MM-dd HH:mm:ss.SSS"}         }   }, max_merge_mb=5, max_cached_mb=30, refresh_seconds=60, ram_buffer_mb=64, class_name=com.stratio.cassandra.lucene.Index, target=stratio_col}
    at com.datastax.driver.core.exceptions.InvalidConfigurationInQueryException.copy(InvalidConfigurationInQueryException.java:37)
    at com.datastax.driver.core.exceptions.InvalidConfigurationInQueryException.copy(InvalidConfigurationInQueryException.java:27)
    at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
    at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
    at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
    at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:43)
    at com.diehl.dcs.stratus.service_database.services.impl.DatabaseServiceImpl.setupTables(DatabaseServiceImpl.java:832)
    at com.diehl.dcs.stratus.service_database.services.impl.DatabaseServiceImpl.init(DatabaseServiceImpl.java:291)
    at com.diehl.dcs.stratus.service_database.services.impl.DatabaseServiceImpl.init(DatabaseServiceImpl.java:273)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)
    at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:311)
    at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:134)

The part of the code referenced looks like this (and is working in application):

session.execute("CREATE CUSTOM INDEX IF NOT EXISTS status_index ON " + TABLE_STATE_SERIES + " (stratio_col) USING 'com.stratio.cassandra.lucene.Index' "
                + "WITH OPTIONS = { "
                + " 'refresh_seconds'       : '60', "
                + " 'ram_buffer_mb'         : '64', "
                + " 'max_merge_mb'          : '5', "
                + " 'max_cached_mb'         : '30', "
                + " 'schema' : '{ "
                + "     default_analyzer : \"english\", "
                + "     fields : { "
                + "         customerid      : {type : \"string\"}, "
                + "         businessstate   : {type : \"string\"}, "
                + "         gatewayid       : {type : \"string\"}, "
                + "         start           : {type : \"date\", pattern: \"yyyy-MM-dd HH:mm:ss.SSS\"}, "
                + "         end             : {type : \"date\", pattern: \"yyyy-MM-dd HH:mm:ss.SSS\"}, "
                + "         duration        : {type : \"date_range\", from : \"start\", to : \"end\", pattern: \"yyyy-MM-dd HH:mm:ss.SSS\"} "
                + "     } "
                + " }' "
                + "}");

We are using the following versions:

<dependency>
    <groupId>com.datastax.cassandra</groupId>
    <artifactId>cassandra-driver-core</artifactId>
    <version>3.2.0</version>
</dependency>

<dependency>
    <groupId>org.apache.cassandra</groupId>
    <artifactId>cassandra-all</artifactId>
    <version>3.9</version>
    <scope>test</scope>
</dependency>

<dependency>
    <groupId>com.stratio.cassandra</groupId>
    <artifactId>cassandra-lucene-index-plugin</artifactId>
    <version>3.9.1</version>
    <scope>test</scope>
</dependency>

Please note, that both the dependency to cassandra-all and the dependency to the lucene-plugin are only with scope test, because I only need em here for the integration test, as in production both are used within the cassandra node, which are completely separate....

Any help would be appreciated.

Cheers Merkaban

[ealonsodb]

Hi @Merkaban:

Could you please test it with cassandra-lucene-index:3.9.7 to discard known issues? You need to reindex the data.

Always, while looking for debugging errors in cassandra-lucene-index, it is useful to activate DEBUG or TRACE logs and inspect them.

In a normal cassandra installation you just need to add a new line in conf/logback.xml.

....
  <logger name="org.apache.cassandra" level="DEBUG"/>
  <logger name="com.thinkaurelius.thrift" level="ERROR"/>
  <logger name="com.stratio.cassandra" level="DEBUG"/> <!-- this line --> 
</configuration>

I think you can get this working just placing edited logback.xml in test-accesible-resources folder.

Hope this helps Looking forward your answer

[Merkaban]

Hi @ealonsodb ,

thank you for your fast answer. Incrementing the version to 3.9.7 did indeed solve the issue. Sadly, right now I will have different versions running in the tests, but that's another story. Again, thanks for your help. Cheers Merkaban

[ealonsodb]

Your welcome @Merkaban