support ElasticSearch 7.*

[templth]

Hello,

In the same way than https://github.com/dokku/dokku-elasticsearch/issues/61 and https://github.com/dokku/dokku-elasticsearch/issues/61#issuecomment-395429548 for ElasticSearch 5. and 6. , I try to use ElasticSearch 7.*.

I found the list of images here: https://www.docker.elastic.co/#.

I tried the following command to create the index:

$ export ELASTICSEARCH_IMAGE_VERSION="7.0.1"
$ dokku elasticsearch:create myindex

but got the following error:

7.0.1: Pulling from library/elasticsearch
8ba884070f61: Already exists 
d22435c65c5b: Pull complete 
779ea19c1b10: Pull complete 
c19ffd8dfed9: Pull complete 
8ef2c51b97f1: Pull complete 
3abad9369e16: Pull complete 
1192f02dde85: Pull complete 
Digest: sha256:6dc77ee54c158808718a769fdfbbadc37578541a87c7f99fc65a8f809183f074
Status: Downloaded newer image for elasticsearch:7.0.1
=====> Extracting config files
dd84950c528e92648bc795747fac420b29ce84c88bf77c1f5749d4f07aaec226
       Waiting for container to be ready
ERROR: unable to connect

Thanks for your help

[templth]

After having investigated a bit more, I found an error message in traces with a docker logs:

"type": "server", "timestamp": "2019-07-12T06:40:56,872+0000", "level": "INFO", "component": "o.e.b.BootstrapChecks", "cluster.name": "docker-cluster", "node.name": "0142f353f6f2",  "message": "bound or publishing to a non-loopback address, enforcing bootstrap checks"  }
ERROR: [1] bootstrap checks failed
[1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

I configured the Elasticseach config file (/var/lib/dokku/services/elasticsearch/myindex/config/elasticsearch.yml) with the following:

node.name: node-1
cluster.name: docker-cluster
network.host: 0.0.0.0
cluster.initial_master_nodes:
  - node-1

I also checked all the points of this link: https://github.com/dokku/dokku-elasticsearch/issues/53#issuecomment-435880511

I did then a restart:

dokku elasticsearch:restart myindex

After that ElasticSearch is up and running... It seems to be linked to new mechanisms regarding clusters with version 7.

[LeviSchuck]

Thank you so much @templth I was wondering why my dokku machine was getting saturated to 100% for over an hour. Finally realized it was restarting over and over and then found this message.

[LeviSchuck]

It appears to use version 7, ELASTICSEARCH_HOSTS must be set on linked apps, the ELASTICSEACH_URL is no longer used. https://www.elastic.co/guide/en/kibana/current/docker.html

[templth]

@LeviSchuck thanks for the hint!

[josegonzalez]

@LeviSchuck thats for kibana, not for elasticsearch.

[LeviSchuck]

As a plugin, it should provide the expected environment variables that are linked to it.

Previously, the kibana application expected ELASTICSEACH_URL which this plugin provides, but with version 7.2.0 expectations across the elastic stack seems to have shifted to _HOSTS

[josegonzalez]

The datastore plugins all export a ${SOMETHING}_URL environment variable. This is the DSN format, which is widely supported across most frameworks and applications. Kibana did something different here, and while it might be nice to export it in this specific case, this datastore plugin was not specifically made to support kibana, so I'd rather they actually go back to supporting the old method.

DSNs also support multiple hosts - mongodb actually promotes this pattern for replicasets via the comma , separator - so its not like they couldn't do this with the pre-existing method.

[LeviSchuck]

You've got a good point.

[metakermit]

Hey, we're trying out these instructions with ElasticSearch 7.7.0, but we're running into some errors. Is there perhaps a list somewhere of the versions that dokku-elasticsearch officially supports?

elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fe8c016fc50>: Failed to establish a new connection: [Errno -2] Name or service not known) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fe8c016fc50>: Failed to establish a new connection: [Errno -2] Name or service not known)

We think it's a Dokku-related problem, because we have the same codebase running on Heroku with their Bonsai ElasticSearch plugin.

[metakermit]

OK an update for future readers – following the instructions above I fixed the issue by installing ElasticSearch version 7.2.0. The steps I took are (adapt the app / elastic search instance names):

docker pull elasticsearch:7.2.0

# unlink & delete any old versions
dokku elasticsearch:unlink otvoreniakties otvoreniakti
dokku elasticsearch:destroy otvoreniakties

# install & link the new version
export ELASTICSEARCH_IMAGE_VERSION="7.2.0"
dokku elasticsearch:create otvoreniakties --image-version 7.2.0
# make the change in the config file like suggested above:
emacs /var/lib/dokku/services/elasticsearch/otvoreniakties/config/elasticsearch.yml
dokku elasticsearch:restart otvoreniakties
dokku elasticsearch:link otvoreniakties otvoreniakti

[josegonzalez]

This is done in the plugin as of 1.14.0. Thanks for the initial debugging that made this possible.