noamt / elasticsearch-grails-plugin

The revived ElasticSearch grails plugin
Based on Graeme Rocher initial stub. Note that it is still in early stage.
Other
63 stars 83 forks source link

Setting index name fails in 1.4.1 with grails 3.2.x #212

Closed tobisinghania closed 6 years ago

tobisinghania commented 6 years ago

Using Grails 3.2.3 with the plugin version 1.4.1 together with elastic search 5.4.1 application startup fails with the following error, if an index name is set:

...
Caused by: java.lang.IllegalArgumentException: unknown setting [index.name] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:293)
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:256)
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:246)
    at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService.onlyCreateIndex(MetaDataCreateIndexService.java:221)
    at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService.createIndex(MetaDataCreateIndexService.java:201)
    at org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.masterOperation(TransportCreateIndexAction.java:81)
    at org.elasticsearch.action.admin.indices.create.TransportCreateIndexAction.masterOperation(TransportCreateIndexAction.java:39)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction.masterOperation(TransportMasterNodeAction.java:87)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$2.doRun(TransportMasterNodeAction.java:166)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.doStart(TransportMasterNodeAction.java:163)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.start(TransportMasterNodeAction.java:126)
    at org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:104) 5.4.1
    at org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:54)
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:170)
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:142)
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:84)
    at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:83)
    at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:72)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:408)
    at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1256)
    at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80)
    at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54)
    at org.elasticsearch.action.ActionRequestBuilder$execute.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
    at grails.plugins.elasticsearch.ElasticSearchAdminService$_createIndex_closure8.doCall(ElasticSearchAdminService.groovy:196)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1426)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:57)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at grails.plugins.elasticsearch.ElasticSearchHelper.withElasticSearch(ElasticSearchHelper.groovy:10)
    at grails.plugins.elasticsearch.ElasticSearchHelper$withElasticSearch.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at grails.plugins.elasticsearch.ElasticSearchAdminService.createIndex(ElasticSearchAdminService.groovy:188)
    at grails.plugins.elasticsearch.ElasticSearchAdminService$createIndex.callCurrent(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:182)
    at grails.plugins.elasticsearch.ElasticSearchAdminService.createIndex(ElasticSearchAdminService.groovy:208)
    at grails.plugins.elasticsearch.mapping.SearchableClassMappingConfigurator.createIndexWithMappings(SearchableClassMappingConfigurator.groovy:181)
    at grails.plugins.elasticsearch.mapping.SearchableClassMappingConfigurator.access$0(SearchableClassMappingConfigurator.groovy)
    at grails.plugins.elasticsearch.mapping.SearchableClassMappingConfigurator$_installMappings_closure3.doCall(SearchableClassMappingConfigurator.groovy:130)
Disconnected from the target VM, address: '127.0.0.1:39388', transport: 'socket'
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1426)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
    at groovy.lang.Closure.call(Closure.java:414)
    at groovy.lang.Closure.call(Closure.java:430)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2030)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2015)
    at org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2068)
    at grails.plugins.elasticsearch.mapping.SearchableClassMappingConfigurator.installMappings(SearchableClassMappingConfigurator.groovy:116)
    at grails.plugins.elasticsearch.mapping.SearchableClassMappingConfigurator.configureAndInstallMappings(SearchableClassMappingConfigurator.groovy:56)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springsource.loaded.ri.ReflectiveInterceptor.jlrMethodInvoke(ReflectiveInterceptor.java:1426)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1711)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1650)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1579)
    ... 21 common frames omitted
:bootRun FAILED

Example for the configuration:

elasticSearch:
  index:
    name: esdemo_test

As far as I understand, the index.name config is only used by the plugin but not by elastic search itself but when a configuration for the index name is specified, the config key seems to be forwarded to elastic search's index settings (together with other configurations like numberOfReplicas). Elastic search itself rejects the settings, as no settings value for index.name is expected and the application startup fails.

Grails version: 3.2.3 Plugin version: 1.4.1 Elasticsearch version: 5.4.1

verglor commented 6 years ago

I have the same problem with version 2.4.0.RC1 in Grails 3.3.2

puneetbehl commented 6 years ago

From Elasticsearch 5.0 on only selected settings like for instance index.codec can be set on the node level. Please see more information here

Please let me know if this is not the case.

verglor commented 6 years ago

Hi @puneetbehl, index.name configuration should not be passed to elasticsearch. It is just alternative name of index (instead of package name) and should be used by plugin internally when creating index. Please see PR#56 - especially getIndexName() method.

puneetbehl commented 6 years ago

@verglor I am sorry for the confusion earlier but the issue is related to index.name passed in indexSettings to elasticSearchAdminService.createIndex(indexName, indexSettings, mappings) method.

I think index default settings should be passed as index.settings.numberOfReplicas in the config rather than index.numberOfReplicas so that there is no conflict between indexName and settings. Right now, the only solution I see is to remove 'index.nameexplicitly before passing thesettingsto createIndexRequest inElasticSearchAdminService.groovy` class.

 void createIndex(String index, Map settings=null, Map<String, Map> esMappings = [:]) {
        LOG.debug "Creating index ${index} ..."

        elasticSearchHelper.withElasticSearch { Client client ->
            CreateIndexRequestBuilder builder = client.admin().indices().prepareCreate(index)
            if (settings) {
->                builder.setSettings(settings)
            }
            esMappings.each { String type, Map elasticMapping ->
                builder.addMapping(type, elasticMapping)
            }
            builder
            builder.execute().actionGet()
        }
    }
puneetbehl commented 6 years ago

I will try to release a new version soon with the fix.

verglor commented 6 years ago

I strongly vote for separating default index settings (passed directly to elasticsearch) to separate config key (e.g. elasticSearch.index.settings as suggested by @puneetbehl) to prevent confusion.

puneetbehl commented 6 years ago

Just an update that this has been implemented in branch v2.4.0.RC2 and soon be released and meanwhile, I am finding some challenges with updating specs in class SearchableClassMappingSpec