search

dockhippie / elasticsearch

Docker images for elasticsearch
MIT License
10 stars 7 forks source link

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] #64

Closed tchiaspko closed 11 months ago

tchiaspko commented 11 months ago

Here is my docker-compose.yml

version: '3.1'
services:
 elastic_search:
  image: webhippie/elasticsearch:5.6-arm64
  container_name: es01
  environment:
   - discovery.type=single-node
   - node.name=es01
   - bootstrap.memory_lock=true
   - ELASTICSEARCH_MAX_MAP_COUNT=262144
  mem_limit: 2g
  ports:
   - 9200:9200

docker-compose up

~/es5.6  docker-compose up
[+] Running 1/0
 ✔ Container es01  Recreated                                                                                                                                                                                                0.0s
Attaching to es01
es01  | Enforcing max mapcount...
es01  | sysctl: error setting key 'vm.max_map_count': Read-only file system
es01 exited with code 1
 ~/es5.6 

If I don't include ELASTICSEARCH_MAX_MAP_COUNT=262144, then I got the following

 ~/es5.6  docker-compose up
[+] Running 1/0
 ✔ Container es01  Recreated                                                                                                                                                                                                0.0s
Attaching to es01
es01  | Enforcing open filedescriptors...
es01  | > writing server config
es01  | {"level":"debug","time":"2023-11-02T03:32:50Z","msg":"starting gomplate"}
es01  | {"level":"debug","version":"3.11.5","build":"ea986e74","time":"2023-11-02T03:32:50Z","msg":"config is:\n---\ninputFiles: [/etc/templates/elasticsearch.yml.tmpl]\noutputFiles: [/etc/elasticsearch/elasticsearch.yml]\n"}
es01  | {"level":"debug","templatesRendered":1,"errors":0,"duration":0.000325916,"time":"2023-11-02T03:32:50Z","msg":"completed rendering"}
es01  | > writing jvm options
es01  | {"level":"debug","time":"2023-11-02T03:32:50Z","msg":"starting gomplate"}
es01  | {"level":"debug","version":"3.11.5","build":"ea986e74","time":"2023-11-02T03:32:50Z","msg":"config is:\n---\ninputFiles: [/etc/templates/jvm.options.tmpl]\noutputFiles: [/etc/elasticsearch/jvm.options]\n"}
es01  | {"level":"debug","templatesRendered":1,"errors":0,"duration":0.000132625,"time":"2023-11-02T03:32:50Z","msg":"completed rendering"}
es01  | Fixing base permissions...
es01  | Fixing config permissions...
es01  | Deleting gc logfiles...
es01  | > starting elasticsearch service
es01  | [2023-11-02T03:32:51,245][INFO ][o.e.n.Node               ] [c1ed74e16d75] initializing ...
es01  | [2023-11-02T03:32:51,285][INFO ][o.e.e.NodeEnvironment    ] [c1ed74e16d75] using [1] data paths, mounts [[/var/lib/elasticsearch (/dev/disk/by-label/data-volume)]], net usable_space [687.5mb], net total_space [58.7gb], spins? [possibly], types [ext4]
es01  | [2023-11-02T03:32:51,286][INFO ][o.e.e.NodeEnvironment    ] [c1ed74e16d75] heap size [247.5mb], compressed ordinary object pointers [true]
es01  | [2023-11-02T03:32:51,286][INFO ][o.e.n.Node               ] [c1ed74e16d75] node name [c1ed74e16d75], node ID [Q4oL60YiTrK2ylrqZhd0jw]
es01  | [2023-11-02T03:32:51,286][INFO ][o.e.n.Node               ] [c1ed74e16d75] version[5.6.16], pid[8], build[3a740d1/2019-03-13T15:33:36.565Z], OS[Linux/5.10.109-0-virt/aarch64], JVM[IcedTea/OpenJDK 64-Bit Server VM/1.8.0_372/25.372-b07]
es01  | [2023-11-02T03:32:51,286][INFO ][o.e.n.Node               ] [c1ed74e16d75] JVM arguments [-XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -XX:-OmitStackTraceInFastThrow, -XX:+HeapDumpOnOutOfMemoryError, -Djava.io.tmpdir=/tmp, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Dlog4j.skipJansi=true, -XX:HeapDumpPath=data, -XX:ErrorFile=/var/lib/elasticsearch/logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/lib/elasticsearch/logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.plugins=/var/lib/elasticsearch/plugins, -Des.path.script=/var/lib/elasticsearch/scripts, -Des.path.work=/var/lib/elasticsearch/work, -Des.scripting.exception_for_missing_value=true, -Xms256m, -Xmx256m, -Xss1m, -Des.path.home=/usr/share/elasticsearch]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [aggs-matrix-stats]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [ingest-common]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [lang-expression]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [lang-groovy]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [lang-mustache]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [lang-painless]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [parent-join]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [percolator]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [reindex]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [transport-netty3]
es01  | [2023-11-02T03:32:51,658][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] loaded module [transport-netty4]
es01  | [2023-11-02T03:32:51,659][INFO ][o.e.p.PluginsService     ] [c1ed74e16d75] no plugins loaded
es01  | [2023-11-02T03:32:52,212][INFO ][o.e.d.DiscoveryModule    ] [c1ed74e16d75] using discovery type [zen]
es01  | [2023-11-02T03:32:52,411][INFO ][o.e.n.Node               ] [c1ed74e16d75] initialized
es01  | [2023-11-02T03:32:52,411][INFO ][o.e.n.Node               ] [c1ed74e16d75] starting ...
es01  | [2023-11-02T03:32:52,514][INFO ][o.e.t.TransportService   ] [c1ed74e16d75] publish_address {172.22.0.2:9300}, bound_addresses {0.0.0.0:9300}
es01  | [2023-11-02T03:32:52,520][INFO ][o.e.b.BootstrapChecks    ] [c1ed74e16d75] bound or publishing to a non-loopback address, enforcing bootstrap checks
es01  | ERROR: [1] bootstrap checks failed
es01  | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
es01  | [2023-11-02T03:32:52,526][INFO ][o.e.n.Node               ] [c1ed74e16d75] stopping ...
es01  | [2023-11-02T03:32:52,549][INFO ][o.e.n.Node               ] [c1ed74e16d75] stopped
es01  | [2023-11-02T03:32:52,549][INFO ][o.e.n.Node               ] [c1ed74e16d75] closing ...
es01  | [2023-11-02T03:32:52,552][INFO ][o.e.n.Node               ] [c1ed74e16d75] closed
tboerger commented 11 months ago

https://github.com/spujadas/elk-docker/issues/99#issuecomment-399751979 says it pretty well how you got to define the value these days. Back when I created the images the current solution worked, but looks like that has changed.

tchiaspko commented 11 months ago

Thanks tboerger, I figured it out last night. I though it has to be set inside the docker

I am on a Mac using colima instead of Docker Desktop so instead of running the sysctl on my Mac, I have to do it inside the colima VM like this 

colima ssh -- sudo sysctl -w vm.max_map_count=262144

Running the sysctl on my Mac doesn't work. 

sudo sysctl -w vm.max_map_count=262144
Password:
sysctl: unknown oid 'vm.max_map_count'
and-semakin commented 10 months ago

Here is the answer that worked for me: https://stackoverflow.com/a/77431685/10650942

I'm on M1 Mac using MacOS Sonoma 14.1 and Docker Desktop 4.25.0.

Summary: