search

TheHive-Project / CortexDocs

Documentation of Cortex
https://docs.strangebee.com/cortex
GNU Affero General Public License v3.0
170 stars 122 forks source link

ElasticSearch Docker instructions don't work on Ubuntu 20.04 #77

Open aaronkaplan opened 3 years ago

aaronkaplan commented 3 years ago

Copy & pasting and modifying the ELK instructions:

cat elastic-install.sh
docker run \
  --name elasticsearch \
  --hostname elasticsearch \
  --rm \
  --publish 127.0.0.1:9200:9200 \
  --volume /data/elastic:/usr/share/elasticsearch/data \
    -e "http.host=0.0.0.0" \
    -e "xpack.security.enabled=false" \
    -e "cluster.name=hive" \
  -e "script.inline=true" \
  -e "thread_pool.search.queue_size=100000" \
    docker.elastic.co/elasticsearch/elasticsearch:7.9.1

Gives (note the java.lang.IllegalArgumentException: unknown setting [script.inline] please check that any required plugins are installed, or check the breaking changes documentation for removed settings", part ):

aaron@NANU:~/work/projects/cortex$ sh elastic-install.sh 
Unable to find image 'docker.elastic.co/elasticsearch/elasticsearch:7.9.1' locally
7.9.1: Pulling from elasticsearch/elasticsearch
f1feca467797: Pull complete 
dcfca94e7428: Pull complete 
d2bf8b28bdf5: Pull complete 
5efd10fdc328: Pull complete 
71948c71bf56: Pull complete 
3d79fd8021d0: Pull complete 
3561742200e5: Pull complete 
2811408f56d0: Pull complete 
cb5a557b51ee: Pull complete 
Digest: sha256:0a5308431aee029636858a6efe07e409fa699b02549a78d7904eb931b8c46920
Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.9.1
{"type": "server", "timestamp": "2021-01-17T08:32:26,812Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "hive", "node.name": "elasticsearch", "message": "version[7.9.1], pid[7], build[default/docker/083627f112ba94dffc1232e8b42b73492789ef91/2020-09-01T21:22:21.964974Z], OS[Linux/5.8.0-36-generic/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/14.0.1/14.0.1+7]" }
{"type": "server", "timestamp": "2021-01-17T08:32:26,814Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "hive", "node.name": "elasticsearch", "message": "JVM home [/usr/share/elasticsearch/jdk]" }
{"type": "server", "timestamp": "2021-01-17T08:32:26,814Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "hive", "node.name": "elasticsearch", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/elasticsearch-13677390955176096374, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,834Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [aggs-matrix-stats]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,834Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [analysis-common]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,834Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [constant-keyword]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,834Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [flattened]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,834Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [frozen-indices]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,834Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [ingest-common]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,834Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [ingest-geoip]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,834Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [ingest-user-agent]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [kibana]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [lang-expression]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [lang-mustache]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [lang-painless]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [mapper-extras]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [parent-join]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [percolator]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [rank-eval]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,835Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [reindex]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [repository-url]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [search-business-rules]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [searchable-snapshots]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [spatial]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [tasks]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [transform]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [transport-netty4]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [vectors]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [wildcard]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,836Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-analytics]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-async]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-async-search]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-autoscaling]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-ccr]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-core]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-data-streams]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-deprecation]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-enrich]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-eql]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-graph]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,837Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-identity-provider]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-ilm]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-logstash]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-ml]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-monitoring]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-ql]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-rollup]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-security]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-sql]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-stack]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,838Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-voting-only-node]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,839Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "loaded module [x-pack-watcher]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,839Z", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "hive", "node.name": "elasticsearch", "message": "no plugins loaded" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,863Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "hive", "node.name": "elasticsearch", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/vgubuntu-root)]], net usable_space [1.5tb], net total_space [1.7tb], types [ext4]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,863Z", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "hive", "node.name": "elasticsearch", "message": "heap size [1gb], compressed ordinary object pointers [true]" }
{"type": "server", "timestamp": "2021-01-17T08:32:27,884Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "hive", "node.name": "elasticsearch", "message": "node name [elasticsearch], node ID [b06lrNuHTd6PRWCIceZZKw], cluster name [hive]" }
{"type": "server", "timestamp": "2021-01-17T08:32:29,075Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "hive", "node.name": "elasticsearch", "message": "uncaught exception in thread [main]", 
"stacktrace": ["org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [script.inline] please check that any required plugins are installed, or check the breaking changes documentation for removed settings",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) ~[elasticsearch-cli-7.9.1.jar:7.9.1]",
"at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.9.1.jar:7.9.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.9.1.jar:7.9.1]",
"Caused by: java.lang.IllegalArgumentException: unknown setting [script.inline] please check that any required plugins are installed, or check the breaking changes documentation for removed settings",
"at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:544) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:489) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:460) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:431) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:149) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.node.Node.<init>(Node.java:385) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.node.Node.<init>(Node.java:277) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:227) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.9.1.jar:7.9.1]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.9.1.jar:7.9.1]",
"... 6 more"] }
uncaught exception in thread [main]
java.lang.IllegalArgumentException: unknown setting [script.inline] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:544)
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:489)
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:460)
    at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:431)
    at org.elasticsearch.common.settings.SettingsModule.<init>(SettingsModule.java:149)
    at org.elasticsearch.node.Node.<init>(Node.java:385)
    at org.elasticsearch.node.Node.<init>(Node.java:277)
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:227)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393)
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170)
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161)
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127)
    at org.elasticsearch.cli.Command.main(Command.java:90)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92)
For complete error details, refer to the log at /usr/share/elasticsearch/logs/hive.log
aaronkaplan commented 3 years ago

@jeromeleonard any comments on this issue?

aaronkaplan commented 3 years ago

Seems that this is a common problem: https://discuss.elastic.co/t/cant-start-elasticsearch-service-unknwon-setting-script-inline/167886