compose up error

[roccotocco]

Hello when im running the command docker compose up i'm getting the following error:

unexpected character "." in variable name near "node.name=elasticsearch\ncluster.name=${CLUSTER_NAME}\ncluster.initial_master_nodes=elasticsearch\nELASTIC_PASSWORD=${ELASTIC_PASSWORD}\nbootstrap.memory_lock=true\nxpack.security.enabled=true\nxpack.security.http.ssl.enabled=true\nxpack.security.http.ssl.key=certs/elasticsearch/elasticsearch.key\nxpack.security.http.ssl.certificate=certs/elasticsearch/elasticsearch.crt\nxpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt\nxpack.security.transport.ssl.enabled=true\nxpack.security.transport.ssl.key=certs/elasticsearch/elasticsearch.key\nxpack.security.transport.ssl.certificate=certs/elasticsearch/elasticsearch.crt\nxpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt\nxpack.security.transport.ssl.verification_mode=certificate\nxpack.license.self_generated.type=${LICENSE}\n"

what's wrong? Thanks

[fabrizio-s]

Hey. I just tested it and it sets up everything correctly. I am using docker version 26.0.1 and docker compose version 2.26.1 on Arch Linux. Did you change any of the configs besides the passwords?

[roccotocco]

Docker version 26.1.0, build 9714adc and Docker Compose version v2.3.3 on Ubuntu 22.04.4 LTS as instructions i set only ELASTIC_PASSWOR and KIBANA_PASSWORD into .env..... Thanks

[fabrizio-s]

Hey. So I managed to replicate your issue on a fresh install of Ubuntu 22.04.4 LTS. After a fresh install, I installed docker and docker compose as described in these 2 articles:

Docker: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-22-04

Docker compose: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-ubuntu-22-04

Then I cloned this repo and ran docker compose up and was able to replicate the issue.

To solve the issue I simply upgraded the docker compose version. So, after another fresh install of Ubuntu, I installed a more recent version of docker compose. So, when the second article instructs you to download version 2.3.3 of docker compose, I simply replaced "v2.3.3" with "v2.26.1" in the url like so:

curl -SL https://github.com/docker/compose/releases/download/v2.26.1/docker-compose-linux-x86_64 -o ~/.docker/cli-plugins/docker-compose

Then I proceeded with the rest of the instructions on the second article, and cloned this repo again and docker compose up worked. Hope this was helpful!

[roccotocco]

Thank you so much, after run the docker compose up it stopping and giving this error:

flowmon-elasticsearch | {"@timestamp":"2024-04-23T13:56:08.083Z", "log.level": "INFO", "message":"bound or publishing to a non-loopback address, enforcing bootstrap checks", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.BootstrapChecks","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"flowmon"} flowmon-elasticsearch | {"@timestamp":"2024-04-23T13:56:08.089Z", "log.level":"ERROR", "message":"node validation exception\n[1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch. For more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.11/bootstrap-checks.html]\nbootstrap check failure [1] of [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]; for more information see [https://www.elastic.co/guide/en/elasticsearch/reference/8.11/_maximum_map_count_check.html]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"flowmon"} flowmon-elasticsearch | ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/flowmon.log flowmon-elasticsearch | {"@timestamp":"2024-04-23T13:56:08.093Z", "log.level": "WARN", "message":"unexpected exception while waiting for http server to close", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"Thread-0","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"flowmon","error.type":"java.util.concurrent.ExecutionException","error.message":"java.lang.IllegalStateException: Can't move to stopped state when not started","error.stack_trace":"java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Can't move to stopped state when not started\n\tat java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)\n\tat java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)\n\tat org.elasticsearch.server@8.11.4/org.elasticsearch.node.Node.prepareForClose(Node.java:1776)\n\tat org.elasticsearch.server@8.11.4/org.elasticsearch.bootstrap.Elasticsearch.shutdown(Elasticsearch.java:468)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: java.lang.IllegalStateException: Can't move to stopped state when not started\n\tat org.elasticsearch.server@8.11.4/org.elasticsearch.common.component.Lifecycle.canMoveToStopped(Lifecycle.java:128)\n\tat org.elasticsearch.server@8.11.4/org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:73)\n\tat org.elasticsearch.server@8.11.4/org.elasticsearch.node.Node.lambda$prepareForClose$59(Node.java:1768)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)\n\t... 1 more\n"} flowmon-elasticsearch | {"@timestamp":"2024-04-23T13:56:08.094Z", "log.level": "INFO", "message":"stopping ...", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"Thread-0","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"flowmon"} flowmon-elasticsearch | {"@timestamp":"2024-04-23T13:56:08.110Z", "log.level": "INFO", "message":"stopped", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"Thread-0","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"flowmon"} flowmon-elasticsearch | {"@timestamp":"2024-04-23T13:56:08.110Z", "log.level": "INFO", "message":"closing ...", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"Thread-0","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"flowmon"} flowmon-elasticsearch | {"@timestamp":"2024-04-23T13:56:08.121Z", "log.level": "INFO", "message":"closed", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"Thread-0","log.logger":"org.elasticsearch.node.Node","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"flowmon"} flowmon-elasticsearch | {"@timestamp":"2024-04-23T13:56:08.123Z", "log.level": "INFO", "message":"Native controller process has stopped - no new native processes can be started", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"ml-cpp-log-tail-thread","log.logger":"org.elasticsearch.xpack.ml.process.NativeController","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"flowmon"} flowmon-elasticsearch | flowmon-elasticsearch | ERROR: Elasticsearch exited unexpectedly, with exit code 78 dependency failed to start: container flowmon-elasticsearch is unhealthy

[roccotocco]

solved..... thank you so much, a questio which kibana dashboard to import ? Thanks

[fabrizio-s]

The logs you posted seem to be a memory related issue. Elastic stack is very memory hungry unfortunately.

And thanks for pointing it out. I have updated the README of this repo with additional information about the kibana dashboard to use. I've used this one: kibana-8.2.x-flow-codex.ndjson. And instructions on how to import it can be found on this video at the timestamp 6:17: https://youtu.be/OB7tWPthBI0?t=378

[roccotocco]

once again thank you.... all working correctly now... anyway if you can help i have last question.. i tried to enable maxmind but when starting im getting the error bout missing GeoLite2-ASN.mmdb and GeoLite2-City.mmdb into the container... what to do ?

[fabrizio-s]

No problem! Glad I was able to help!

Unfortunately I never got around to setting up MaxMind since that goes beyond my basic needs, so I do not know how to help you configuring that. But I do know there is another video here from Elastiflow which explains how it can be configured: https://www.youtube.com/watch?v=O4l4QiGKld0

The process seems to involve creating an account on MaxMind and changing some configurations in the flow-collector.env file. Hope this helps.