search

docker / kitematic

Visual Docker Container Management on Mac & Windows
https://kitematic.com
Apache License 2.0
12.23k stars 1.41k forks source link

ElasticSearch 7+ "Native controller process has stopped - no new native processes can be started" #5029

Open ediewoelfle opened 5 years ago

ediewoelfle commented 5 years ago

Expected behavior

Logs from a new 6.8.0 ES container: 

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-07-02T17:33:16,932][INFO ][o.e.e.NodeEnvironment    ] [vUjaJui] using [1] data paths, mounts [[/ (overlay)]], net usable_space [42.4gb], net total_space [58.4gb], types [overlay]
[2019-07-02T17:33:16,938][INFO ][o.e.e.NodeEnvironment    ] [vUjaJui] heap size [990.7mb], compressed ordinary object pointers [true]
[2019-07-02T17:33:16,941][INFO ][o.e.n.Node               ] [vUjaJui] node name derived from node ID [vUjaJui4QOShI_nBMJeJ2Q]; set [node.name] to override
[2019-07-02T17:33:16,942][INFO ][o.e.n.Node               ] [vUjaJui] version[6.8.0], pid[1], build[default/docker/65b6179/2019-05-15T20:06:13.172855Z], OS[Linux/4.9.125-linuxkit/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]
[2019-07-02T17:33:16,942][INFO ][o.e.n.Node               ] [vUjaJui] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-3950135291909166212, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.cgroups.hierarchy.override=/, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker]
[2019-07-02T17:33:19,182][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [aggs-matrix-stats]
[2019-07-02T17:33:19,182][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [analysis-common]
[2019-07-02T17:33:19,182][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [ingest-common]
[2019-07-02T17:33:19,182][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [ingest-geoip]
[2019-07-02T17:33:19,182][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [ingest-user-agent]
[2019-07-02T17:33:19,183][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [lang-expression]
[2019-07-02T17:33:19,183][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [lang-mustache]
[2019-07-02T17:33:19,183][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [lang-painless]
[2019-07-02T17:33:19,183][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [mapper-extras]
[2019-07-02T17:33:19,184][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [parent-join]
[2019-07-02T17:33:19,184][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [percolator]
[2019-07-02T17:33:19,185][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [rank-eval]
[2019-07-02T17:33:19,185][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [reindex]
[2019-07-02T17:33:19,186][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [repository-url]
[2019-07-02T17:33:19,187][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [transport-netty4]
[2019-07-02T17:33:19,187][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [tribe]
[2019-07-02T17:33:19,187][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-ccr]
[2019-07-02T17:33:19,188][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-core]
[2019-07-02T17:33:19,188][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-deprecation]
[2019-07-02T17:33:19,188][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-graph]
[2019-07-02T17:33:19,189][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-ilm]
[2019-07-02T17:33:19,189][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-logstash]
[2019-07-02T17:33:19,189][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-ml]
[2019-07-02T17:33:19,190][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-monitoring]
[2019-07-02T17:33:19,190][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-rollup]
[2019-07-02T17:33:19,190][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-security]
[2019-07-02T17:33:19,191][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-sql]
[2019-07-02T17:33:19,191][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-upgrade]
[2019-07-02T17:33:19,191][INFO ][o.e.p.PluginsService     ] [vUjaJui] loaded module [x-pack-watcher]
[2019-07-02T17:33:19,192][INFO ][o.e.p.PluginsService     ] [vUjaJui] no plugins loaded
[2019-07-02T17:33:23,903][INFO ][o.e.x.s.a.s.FileRolesStore] [vUjaJui] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
[2019-07-02T17:33:24,812][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [vUjaJui] [controller/87] [Main.cc@109] controller (64 bit): Version 6.8.0 (Build e6cf25e2acc5ec) Copyright (c) 2019 Elasticsearch BV
[2019-07-02T17:33:26,105][INFO ][o.e.d.DiscoveryModule    ] [vUjaJui] using discovery type [zen] and host providers [settings]
[2019-07-02T17:33:27,495][INFO ][o.e.n.Node               ] [vUjaJui] initialized
[2019-07-02T17:33:27,495][INFO ][o.e.n.Node               ] [vUjaJui] starting ...
[2019-07-02T17:33:27,754][INFO ][o.e.t.TransportService   ] [vUjaJui] publish_address {172.17.0.4:9300}, bound_addresses {0.0.0.0:9300}
[2019-07-02T17:33:27,780][INFO ][o.e.b.BootstrapChecks    ] [vUjaJui] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2019-07-02T17:33:30,876][INFO ][o.e.c.s.MasterService    ] [vUjaJui] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {vUjaJui}{vUjaJui4QOShI_nBMJeJ2Q}{77LyFw04RfSvm5LzMZLTOA}{172.17.0.4}{172.17.0.4:9300}{ml.machine_memory=2095869952, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2019-07-02T17:33:30,891][INFO ][o.e.c.s.ClusterApplierService] [vUjaJui] new_master {vUjaJui}{vUjaJui4QOShI_nBMJeJ2Q}{77LyFw04RfSvm5LzMZLTOA}{172.17.0.4}{172.17.0.4:9300}{ml.machine_memory=2095869952, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {vUjaJui}{vUjaJui4QOShI_nBMJeJ2Q}{77LyFw04RfSvm5LzMZLTOA}{172.17.0.4}{172.17.0.4:9300}{ml.machine_memory=2095869952, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2019-07-02T17:33:30,981][INFO ][o.e.h.n.Netty4HttpServerTransport] [vUjaJui] publish_address {172.17.0.4:9200}, bound_addresses {0.0.0.0:9200}
[2019-07-02T17:33:30,982][INFO ][o.e.n.Node               ] [vUjaJui] started
[2019-07-02T17:33:31,041][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [vUjaJui] Failed to clear cache for realms [[]]
[2019-07-02T17:33:31,184][INFO ][o.e.g.GatewayService     ] [vUjaJui] recovered [0] indices into cluster_state
[2019-07-02T17:33:31,422][INFO ][o.e.c.m.MetaDataIndexTemplateService] [vUjaJui] adding template [.watch-history-9] for index patterns [.watcher-history-9*]
[2019-07-02T17:33:31,456][INFO ][o.e.c.m.MetaDataIndexTemplateService] [vUjaJui] adding template [.watches] for index patterns [.watches*]
[2019-07-02T17:33:31,486][INFO ][o.e.c.m.MetaDataIndexTemplateService] [vUjaJui] adding template [.triggered_watches] for index patterns [.triggered_watches*]
[2019-07-02T17:33:31,531][INFO ][o.e.c.m.MetaDataIndexTemplateService] [vUjaJui] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-6-*]
[2019-07-02T17:33:31,587][INFO ][o.e.c.m.MetaDataIndexTemplateService] [vUjaJui] adding template [.monitoring-es] for index patterns [.monitoring-es-6-*]
[2019-07-02T17:33:31,620][INFO ][o.e.c.m.MetaDataIndexTemplateService] [vUjaJui] adding template [.monitoring-alerts] for index patterns [.monitoring-alerts-6]
[2019-07-02T17:33:31,665][INFO ][o.e.c.m.MetaDataIndexTemplateService] [vUjaJui] adding template [.monitoring-beats] for index patterns [.monitoring-beats-6-*]
[2019-07-02T17:33:31,702][INFO ][o.e.c.m.MetaDataIndexTemplateService] [vUjaJui] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-6-*]
[2019-07-02T17:33:31,795][INFO ][o.e.l.LicenseService     ] [vUjaJui] license [d3424e40-0481-4654-be74-7ebdc85f903b] mode [basic] - valid

Actual behavior

Logs from a new 7.2.0 ES container:

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
{"type": "server", "timestamp": "2019-07-02T17:47:05,402+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "using [1] data paths, mounts [[/ (overlay)]], net usable_space [42.4gb], net total_space [58.4gb], types [overlay]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:05,408+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "heap size [990.7mb], compressed ordinary object pointers [true]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:05,412+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "node name [74175a1d24d7], node ID [7sc8J0dCRJWCC6O2m0EmEw], cluster name [docker-cluster]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:05,413+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "version[7.2.0], pid[1], build[default/docker/508c38a/2019-06-20T15:54:18.811730Z], OS[Linux/4.9.125-linuxkit/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:05,414+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "JVM home [/usr/share/elasticsearch/jdk]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:05,415+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-5100568283784166773, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Des.cgroups.hierarchy.override=/, -Dio.netty.allocator.type=unpooled, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,884+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [aggs-matrix-stats]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,884+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [analysis-common]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,885+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [data-frame]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,886+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [ingest-common]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,888+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [ingest-geoip]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,889+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [ingest-user-agent]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,890+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [lang-expression]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,891+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [lang-mustache]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,892+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [lang-painless]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,893+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [mapper-extras]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,894+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [parent-join]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,894+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [percolator]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,895+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [rank-eval]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,896+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [reindex]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,896+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [repository-url]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,897+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [transport-netty4]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,897+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-ccr]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,898+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-core]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,898+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-deprecation]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,899+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-graph]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,899+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-ilm]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,899+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-logstash]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,900+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-ml]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,900+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-monitoring]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,901+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-rollup]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,901+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-security]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,901+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-sql]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,902+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "loaded module [x-pack-watcher]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:07,903+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "no plugins loaded"  }
{"type": "server", "timestamp": "2019-07-02T17:47:12,944+0000", "level": "INFO", "component": "o.e.x.s.a.s.FileRolesStore", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:13,950+0000", "level": "INFO", "component": "o.e.x.m.p.l.CppLogMessageHandler", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "[controller/103] [Main.cc@110] controller (64 bit): Version 7.2.0 (Build 65aefcbfce449b) Copyright (c) 2019 Elasticsearch BV"  }
{"type": "server", "timestamp": "2019-07-02T17:47:14,782+0000", "level": "DEBUG", "component": "o.e.a.ActionModule", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "Using REST wrapper from plugin org.elasticsearch.xpack.security.Security"  }
{"type": "server", "timestamp": "2019-07-02T17:47:15,268+0000", "level": "INFO", "component": "o.e.d.DiscoveryModule", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "using discovery type [zen] and seed hosts providers [settings]"  }
{"type": "server", "timestamp": "2019-07-02T17:47:16,522+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "initialized"  }
{"type": "server", "timestamp": "2019-07-02T17:47:16,523+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "starting ..."  }
{"type": "server", "timestamp": "2019-07-02T17:47:16,821+0000", "level": "INFO", "component": "o.e.t.TransportService", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "publish_address {172.17.0.4:9300}, bound_addresses {0.0.0.0:9300}"  }
{"type": "server", "timestamp": "2019-07-02T17:47:16,840+0000", "level": "INFO", "component": "o.e.b.BootstrapChecks", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "bound or publishing to a non-loopback address, enforcing bootstrap checks"  }
ERROR: [1] bootstrap checks failed
[1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
{"type": "server", "timestamp": "2019-07-02T17:47:16,861+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "stopping ..."  }
{"type": "server", "timestamp": "2019-07-02T17:47:16,922+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "stopped"  }
{"type": "server", "timestamp": "2019-07-02T17:47:16,923+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "closing ..."  }
{"type": "server", "timestamp": "2019-07-02T17:47:16,953+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "closed"  }
{"type": "server", "timestamp": "2019-07-02T17:47:16,959+0000", "level": "INFO", "component": "o.e.x.m.p.NativeController", "cluster.name": "docker-cluster", "node.name": "74175a1d24d7",  "message": "Native controller process has stopped - no new native processes can be started"  }

Information about the Issue

Every ES container above 6.8.0 fails to start.

Steps to reproduce the behavior

  1. Create a new ES container with a 7.0.0 tag or above.
ediewoelfle commented 5 years ago

adding discovery.type = single-node to the environment variables fixes the issue, but undesirable.

thangchung commented 5 years ago

@ediewoelfle Thank you. You save my day 👍

lupguo commented 5 years ago

My environment is initially equipped with ES cluster, and one node always exits, and then reports exit with 137, the query is a memory problem.

Therefore, I switched to a single ES node and modified my docker-compose.yml configuration. I modified the environment configuration as discovery.type=single-node , and found error report elastic_es01_1 exited with code 1!

My fix: Remove the previous volume binding directory directly(ES Cluster), and run docker again and everything is ok!

My docker-compose.yml: 

version: "3.7"
services:
    es01:
        image: elasticsearch:7.4.2
        environment:
            - node.name=es01
            - cluster.name=es-docker-cluster
            - discovery.type=single-node
        volumes:
            - /data/elastic/data01:/usr/share/elasticsearch/data
        networks:
            - elastic
        ports:
            - 9200:9200
            - 9300:9300
networks:
  elastic:
    driver: bridge
Sysa commented 4 years ago

Looks like you are entered into production mode after explicit network.host: 0.0.0.0 and then, according to your logs, it fails on bootstrap checks :

[1]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

It is required to do proper configuration setup since 7.0+ version. You need to specify at least one of discovery.seed_hosts, discovery.seed_providers or cluster.initial_master_nodes in $ES_HOME/config/elasticsearch.yml

IntrovertedFL commented 4 years ago

Thank you

amitrangari commented 4 years ago

My environment is initially equipped with ES cluster, and one node always exits, and then reports exit with 137, the query is a memory problem.

Therefore, I switched to a single ES node and modified my docker-compose.yml configuration. I modified the environment configuration as discovery.type=single-node , and found error report elastic_es01_1 exited with code 1!

My fix: Remove the previous volume binding directory directly(ES Cluster), and run docker again and everything is ok!

My docker-compose.yml:

version: "3.7"
services:
    es01:
        image: elasticsearch:7.4.2
        environment:
            - node.name=es01
            - cluster.name=es-docker-cluster
            - discovery.type=single-node
        volumes:
            - /data/elastic/data01:/usr/share/elasticsearch/data
        networks:
            - elastic
        ports:
            - 9200:9200
            - 9300:9300
networks:
  elastic:
    driver: bridge

This helped me