voxpupuli / puppet-elasticsearch

Elasticsearch Puppet module
Apache License 2.0
403 stars 479 forks source link

ingest-geoip plugin installs in incorrect location #892

Open vchan2002 opened 6 years ago

vchan2002 commented 6 years ago

It would seem that, when I install the ingest-geoip plugin, it installs the files in /etc/elasticsearch/ingest-geoip, since the puppet process runas as root.

However, since ES-HOME for the instance is set as /etc/elasticsearch/es-01 (I use the instance name of es-01 just to be simple)... it appears that elasticsearch, when it starts is looking for the files at /etc/elasticsearch/es-01/ingest-geoip, and thus it fails to start as a result.

Right now, i'll do a exec workaround to copy the files to the right location, but what suggestions would there be in terms of solving this issue?

java.lang.RuntimeException: java.nio.file.NoSuchFileException: /etc/elasticsearch/es-01/ingest-geoip at org.elasticsearch.ingest.geoip.IngestGeoIpPlugin.getProcessors(IngestGeoIpPlugin.java:74) ~[?:?] at org.elasticsearch.ingest.IngestService.<init>(IngestService.java:58) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.node.Node.<init>(Node.java:354) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.node.Node.<init>(Node.java:245) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:233) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:233) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342) [elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) [elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) [elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) [elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) [elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.6.3.jar:5.6.3] Caused by: java.nio.file.NoSuchFileException: /etc/elasticsearch/es-01/ingest-geoip

elasticsearch::plugin { 'ingest-geoip': module_dir => 'ingest-geoip', instances => 'es-01', }

tylerjl commented 6 years ago

Yep, I can reproduce this as well. Will look into it, thank you for the report 👍

tkuther commented 5 years ago

This issue is still valid.

I have this:

  Elasticsearch::Plugin { instances => [$instance_name], }
  elasticsearch::plugin { 'ingest-geoip': }
  elasticsearch::plugin { 'ingest-user-agent': }

where $instance_name = 'master'

root@es1-new:~# ls -l /etc/elasticsearch/ingest-geoip/
total 70912
-rw-rw---- 1 root elasticsearch  5860547 Oct 28 21:39 GeoLite2-ASN.mmdb
-rw-rw---- 1 root elasticsearch 63478091 Oct 28 21:39 GeoLite2-City.mmdb
-rw-rw---- 1 root elasticsearch  3272131 Oct 28 21:39 GeoLite2-Country.mmdb
root@es1-new:~# ls -l /etc/elasticsearch/master/ingest-geoip/
total 0

So, ingest-geoip doesn't get installed correctly to the instance's ES_PATH_CONF (/etc/elasticsearch/master here)

Filebeat pipelines for system and nginx modules fail to start.