Closed bhanupraveeng closed 4 years ago
@bhanupraveeng can you try with hosts => "elasticsearch:9200"
in your logstash/pipeline/logstash.conf
file? Like I said earlier, it's not recommended to hardcode IP addresses since Docker resolves the name elasticsearch
to the IP of the container, even if its IP changes.
Apart from that I don't see what could be causing this issue. Logstash doesn't report any error before terminating, which is surprising.
Hello Antoineco, The same set-up was working before enabling xpack.security. Even after changing logstash.conf, still the same issue. No logs output for logstash. Please can you advice how can i enable debug logs for logstash? Any command we can add in docker compose?
@bhanupraveeng you forgot the quotes around the username and password in this file: https://github.com/deviantony/docker-elk/blob/master/logstash/pipeline/logstash.conf
You don't need quotes in YAML files, but you do need then inside logstash.conf.
Hello antoineco,
I tried as you said, please find all below updated files. Seems logstash not connecting to elasticsearch and exiting with code 1.. Not sure how to enable logstash debug logs to see exact error. please advice
elasticsearch.yml
cluster.name: "docker-cluster"
network.host: 0.0.0.0
path.repo: ["/opt/hc/esbackup"]
bootstrap.memory_lock: true
discovery.type: single-node
xpack.license.self_generated.type: basic
xpack.security.enabled: true
xpack.monitoring.collection.enabled: true
logstash.yml
http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "http://elasticsearch:9200" ]
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: NE29ubVyRdCVBN6DlES2
logstash.es.output.conf
elasticsearch {
hosts => "elasticsearch:9200"
user => "elastic"
password => "pp9zuwSY06zlDcBlbmMP"
manage_template => false
index => "%{[@metadata][target_index]}"
}
kibana.yml
server.name: kibana
server.host: 0.0.0.0
elasticsearch.hosts: [ "http://elasticsearch:9200" ]
monitoring.ui.container.elasticsearch.enabled: true
elasticsearch.requestTimeout: 120000
elasticsearch.username: kibana
elasticsearch.password: oA8AEX1eKAojZgT7TjDM
Do you have both logstasg-output.conf
and logstash.es.output.conf
? One file is one pipeline, so please make sure logstash/pipeline
contains only one file.
Regarding the debug logs, the procedure is described in the Elastic documentation at https://www.elastic.co/guide/en/logstash/current/logging.html
I have only one file with name "90-elasticsearch-output.conf" which outputs to elasticsearch. I will try to enable debug logging and let you know exact error if outcomes.
Thanks
And if you rename that file to its default name? The naming of the file matters: https://www.elastic.co/guide/en/logstash/current/config-setting-files.html
the same set-up with same config files is working if i disable xpack.security.
@bhanupraveeng without a complete view of your configuration and a way to reproduce your problem this issue will be impossible to solve by us.
Because the problem is not related to Docker or Compose I kindly suggest you to ask for help in a forum dedicated to Elastic products, where you are more likely to find people with a good knowledge of Logstash: https://discuss.elastic.co
Hello,
I set-up this image in production server with single node. Stack came up and working fine. As a part of next step, enabled "xpack.security.enabled: true". Generated random passwords for built-in users and added those users/passwords to config files. post that logstash is not coming up. Kindly check and advice
Enabled logstash security as per below page: https://www.elastic.co/guide/en/logstash/current/ls-security.html#ls-monitoring-user
Docker-compose.yml
Docker setup
elasticsearch.yml
logstash.yml
logstasg-output.conf
kibana.yml
Docker logs