ELK not running as expected

[salizzar]

Hi Russ,

I can't run ELK here. It seems that Elasticsearch is not reachable by Kibana container for some reason. Can you check, please?

Here logs from Elasticsearch container:

[vagrant@docker ~]$ docker logs elk_elasticsearch_1
[2015-11-05 21:28:47,139][INFO ][node                     ] [Umar] version[2.0.0], pid[1], build[de54438/2015-10-22T08:09:48Z]
[2015-11-05 21:28:47,141][INFO ][node                     ] [Umar] initializing ...
[2015-11-05 21:28:47,191][INFO ][plugins                  ] [Umar] loaded [], sites []
[2015-11-05 21:28:47,297][INFO ][env                      ] [Umar] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/sda3)]], net usable_space [90.9gb], net total_space [95.8gb], spins? [possibly], types [xfs]
[2015-11-05 21:28:49,003][INFO ][node                     ] [Umar] initialized
[2015-11-05 21:28:49,003][INFO ][node                     ] [Umar] starting ...
[2015-11-05 21:28:49,101][INFO ][transport                ] [Umar] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2015-11-05 21:28:49,112][INFO ][discovery                ] [Umar] elasticsearch/M06K1SkySPWEoRLqd2TqwQ
[2015-11-05 21:28:52,221][INFO ][cluster.service          ] [Umar] new_master {Umar}{M06K1SkySPWEoRLqd2TqwQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2015-11-05 21:28:52,275][INFO ][http                     ] [Umar] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2015-11-05 21:28:52,275][INFO ][node                     ] [Umar] started
[2015-11-05 21:28:52,280][INFO ][gateway                  ] [Umar] recovered [0] indices into cluster_state

I changed kibana/entrypoint.sh to make sure if Kibana reaches Elasticsearch container as expected, here my code:

#!/usr/bin/env bash

# Wait for the Elasticsearch container to be ready before starting Kibana.
echo "Stalling for Elasticsearch"
while true; do
    nc -q 1 elasticsearch 9200 2>/dev/null && break
    echo "Elasticsearch not ready yeat, waiting."
    sleep 1
done

echo "Starting Kibana"
kibana

Here the output after starting via docker-compose up -d:

[vagrant@docker elk]$ docker logs -f elk_kibana_1
Stalling for Elasticsearch
Elasticsearch not ready yeat, waiting.
Elasticsearch not ready yeat, waiting.
Elasticsearch not ready yeat, waiting.
Elasticsearch not ready yeat, waiting.
(...)

Running inside the container via docker exec -it elk_kibana_1 bash I get the following error:

root@cd361b3cc5d2:/# nc -q 1 elasticsearch 9200
elasticsearch [172.17.0.31] 9200 (?) : Connection refused

Thank you!

[salizzar]

Hi Russ,

Problems related above are detected running Vagrant CentOS machine; Ubuntu have the same behavior.

[russmckendrick]

Sorry, just spotted this. Will take a look now.

[russmckendrick]

Hi Marcelo,

Looks like a combination of the latest Docker release and an update to Elastic Search 2 broke a few things, have just pushed a few fixes which gets everything back up and working. The follow fixes were put in place;

• Make Elasticsearch search listen on ipv4 and not ipv6, I think this was due to Docker update, for some reason Elasticsearch only bound itself to an ipv6 address.
• Update Logstash configuration to use Elasticsearch host
• Update logspount-logstash plugin to remove tags with dots in the title as Elasticsearch 2 doesn't support them

I also added a crafty update to the version of docker-compose so its the latest release so you may want to provision a new Vagrant host.

Thanks,

Russ