search

elastic / elasticsearch-docker

Official Elasticsearch Docker image
Apache License 2.0
789 stars 240 forks source link

Unable to start container with repository-gcs setting added to keystore #206

Closed SubatomicHero closed 5 years ago

SubatomicHero commented 5 years ago

Hi there,

I am building an image using the following Dockerfile:

FROM docker.elastic.co/elasticsearch/elasticsearch:5.4.2

USER root
ENV JQ_VERSION=1.5 \
    JQ_SHA256=c6b3a7d7d3e7b70c6f51b706a3b90bd01833846c54d32ca32f0027f00226ff6d
RUN cd /tmp \
    && curl -o /usr/bin/jq -SL "https://github.com/stedolan/jq/releases/download/jq-$JQ_VERSION/jq-linux64" \
    && echo "$JQ_SHA256  /usr/bin/jq" | sha256sum -c - \
    && chmod +x /usr/bin/jq

# https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_c_customized_image

COPY elasticsearch.yml /usr/share/elasticsearch/config/
COPY encoded /tmp/.

RUN chown elasticsearch:elasticsearch config/elasticsearch.yml \
    && chown elasticsearch:elasticsearch /tmp/encoded \
    && bin/elasticsearch-plugin install io.fabric8:elasticsearch-cloud-kubernetes:5.4.2 \
    && bin/elasticsearch-plugin install repository-gcs

COPY docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh

USER elasticsearch

ENTRYPOINT [ "/usr/local/bin/docker-entrypoint.sh" ]
CMD ["elasticsearch", "-v"]

Here is the elasticsearch.yml config file:

network.host: 0.0.0.0

processors: ${PROCESSORS:}

# minimum_master_nodes need to be explicitly set when bound on a public IP
# set to 1 to allow single node clusters
# Details: https://github.com/elastic/elasticsearch/pull/17288
discovery.zen.minimum_master_nodes: 1

plugin.mandatory: repository-gcs

And the entrypoint script:

#!/bin/bash
set -e

if [ "$1" = 'elasticsearch' ]; then
    echo "Decoding credentials"
    cat /tmp/encoded | base64 -d > /tmp/elasticsearch-backup-service-account.json 
    chown elasticsearch:elasticsearch /tmp/elasticsearch-backup-service-account.json
    mv /tmp/elasticsearch-backup-service-account.json bin/creds.json

    echo "Creating keystore"
    bin/elasticsearch-keystore create

    echo "Adding credentials to keystore"
    cat bin/creds.json | bin/elasticsearch-keystore add-file --stdin gcs.client.gcs1.credentials_file

    echo "Deleting secure files"
    rm -f /tmp/encoded
fi

exec "$@"

When running the container, I get this error:

[2018-10-16T10:57:51,605][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [aggs-matrix-stats]
[2018-10-16T10:57:51,605][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [ingest-common]
[2018-10-16T10:57:51,605][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [lang-expression]
[2018-10-16T10:57:51,605][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [lang-groovy]
[2018-10-16T10:57:51,606][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [lang-mustache]
[2018-10-16T10:57:51,606][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [lang-painless]
[2018-10-16T10:57:51,606][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [percolator]
[2018-10-16T10:57:51,606][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [reindex]
[2018-10-16T10:57:51,606][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [transport-netty3]
[2018-10-16T10:57:51,606][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded module [transport-netty4]
[2018-10-16T10:57:51,607][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded plugin [discovery-kubernetes]
[2018-10-16T10:57:51,607][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded plugin [ingest-geoip]
[2018-10-16T10:57:51,607][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded plugin [ingest-user-agent]
[2018-10-16T10:57:51,607][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded plugin [repository-gcs]
[2018-10-16T10:57:51,608][INFO ][o.e.p.PluginsService     ] [w7B4izN] loaded plugin [x-pack]
[2018-10-16T10:57:53,665][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown secure setting [gcs.client.default.credentials_file] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:127) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.4.2.jar:5.4.2]
Caused by: java.lang.IllegalArgumentException: unknown secure setting [gcs.client.default.credentials_file] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:293) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:256) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:139) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.node.Node.<init>(Node.java:343) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.node.Node.<init>(Node.java:242) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:232) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:232) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:350) ~[elasticsearch-5.4.2.jar:5.4.2]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.4.2.jar:5.4.2]
    ... 6 more

Can anyone direct me in what I am doing wrong please? I have a feeling that the plugin isn't registering the setting correctly or something I can't quite work out

jasontedor commented 5 years ago

This way of configuring the repository-gcs plugin was not added until 6.3.0 yet you're on 5.4.2. You need to refer to the 5.4.2 documentation for how to configure repository-gcs for that version.