Cannot disable autoCreateIndex

[steghio]

Hello,

I'm testing on ES 6.5.3 and 7.1.1

Setting autoCreateIndex=false results in this error:

java.lang.IllegalArgumentException: the [action.auto_create_index] setting value [false] is too restrictive. disable [action.auto_create_index] or set it to [.watches,.triggered_watches,.watcher-history-*]

Setting it to .watches,.triggered_watches,.watcher-history-* is not possible since the parameter expects a boolean value.

Thank you and cheers

[alexcojocaru]

Thanks for reporting the issue, Stefano. I found this related issue reported against ES: https://github.com/elastic/elasticsearch/issues/33369 . I will look into fixing the plugin later.

[alexcojocaru]

This is weird, I cannot reproduce the issue with ES 6.5.3, 7.1.1 or 7.2.0, and I have been using this pom.xml https://gist.github.com/alexcojocaru/61cb2b42d21bf408669bc9a0818cf8f1 . Do you mind sharing your pom file?

[steghio]

Sure here it is:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.6.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <dependencies>
        <!-- Elasticsearch -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-elasticsearch</artifactId>
            <version>3.2.0.M1</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>6.5.3</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>6.5.3</version>
        </dependency>

        <!-- Kafka -->
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>2.2.7.RELEASE</version>
        </dependency>

        <!-- test -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!-- others -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.8</version>
            <scope>provided</scope>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>

            <!-- Integration tests -->
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <version>3.0.0</version>
                <executions>
                    <execution>
                        <id>add-integration-test-source</id>
                        <phase>generate-test-sources</phase>
                        <goals>
                            <goal>add-test-source</goal>
                        </goals>
                        <configuration>
                            <sources>
                                <source>src/it/java</source>
                            </sources>
                        </configuration>
                    </execution>
                    <execution>
                        <id>add-integration-test-resource</id>
                        <phase>generate-test-resources</phase>
                        <goals>
                            <goal>add-test-resource</goal>
                        </goals>
                        <configuration>
                            <resources>
                                <resource>
                                    <directory>src/it/resources</directory>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.22.2</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <!-- https://github.com/alexcojocaru/elasticsearch-maven-plugin -->
            <plugin>
                <groupId>com.github.alexcojocaru</groupId>
                <artifactId>elasticsearch-maven-plugin</artifactId>
                <version>6.13</version>
                <configuration>
                    <version>6.5.3</version>
                </configuration>
                <executions>
                    <execution>
                        <id>start-elasticsearch</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>runforked</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-elasticsearch</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

Some parts of the file I cannot post, but this is basically all of it.

It was actually my issue I verified since I am not using 6.5.3 rather 6.2.2 actually, I checked in the elasticsearch0/lib folder used for the integration testing and I see there is elasticsearch-6.2.2.jar so..

Problem is I am working in a very closed environment and I don't have access to artifacts.elastic.co therefore your plugin cannot download the files it needs directly. I got elasticsearch-6.5.3 manually from our repos but apparently it is not 6.5.3 :)

If you can verify against 6.2.2 you might get same behaviour; seeing on 6.5+ you don't have this problem I think we can close the issue. I currently solved it by leaving index autocreation to the default value (true) and marking all attributes in my POJO with the correct @Field annotation so that indexes can be correctly autocreated and it works for me now.

Thanks again and have a nice day

[alexcojocaru]

Even with ES 6.2.2 and plugin version 6.13, I cannot reproduce the problem.

The ES documentation states that the property can be set to false: https://www.elastic.co/guide/en/elasticsearch/reference/7.2/targz.html#targz-enable-indices https://www.elastic.co/guide/en/elasticsearch/reference/7.2/docs-index_.html#index-creation : last sentence (Finally it can be completely disabled by changing this setting to false.), as well as the second example (PUT _cluster/settings { "persistent": { "action.auto_create_index": "false" } }).

[alexcojocaru]

A workaround to using the autoCreateIndex property on the plugin is not to specify it: https://github.com/alexcojocaru/elasticsearch-maven-plugin/blob/c6328443817db81f087d857f16481a8c65aabaa2/src/main/java/com/github/alexcojocaru/mojo/elasticsearch/v2/ForkedInstance.java#L83-L86 and instead use the instanceSettings property on the plugin, eg:

        <instanceSettings>
            <properties>
                <auto_create_index>.watches,.triggered_watches,.watcher-history-*</auto_create_index>
            </properties>
        </instanceSettings>

Remember that you need multiple properties blocks if you run multiple ES instances in the cluster. The attribute name and value provided using instance settings will be passed to the ES runner as arguments using the -Ename=value syntax: https://github.com/alexcojocaru/elasticsearch-maven-plugin/blob/c6328443817db81f087d857f16481a8c65aabaa2/src/main/java/com/github/alexcojocaru/mojo/elasticsearch/v2/ForkedInstance.java#L90-L93

[alexcojocaru]

Yet another workaround is to use an init script the make use of the Index API to disable the automatic index creation.

[alexcojocaru]

I am going to close this issue for now, since I cannot reproduce it.