spujadas / elk-docker

Elasticsearch, Logstash, Kibana (ELK) Docker image
Other
2.16k stars 908 forks source link

Kibana won’t start after server accidently restart #329

Closed anggara-ajip closed 4 years ago

anggara-ajip commented 4 years ago

hello, I use elk-docker to develop elasticsearch kibana in my office. it worked well. but after the server was accidently restart yesterday, the kibana can't show anymore

even my elasticsearch work well and can be access on localhost:9200 Capture

and here is my indeks status Capture 2

can you help me sir

hacfi commented 4 years ago

@anggara-ajip Two things you can do to debug this:

docker top CONTAINER (for example docker top elk) this should show running a few processing including something like 993 265227 264856 0 Jul07 ? 00:03:54 /opt/kibana/bin/../node/bin/node /opt/kibana/bin/../src/cli -l /var/log/kibana/kibana5.log

and

docker logs CONTAINER (for example docker logs elk) and see if there is anything mentioning kibana not being able to start (or directly view the kibana logs via docker exec elk tail -n 100 /var/log/kibana/kibana5.log).

anggara-ajip commented 4 years ago

@hacfi hello mr I try run this command docker exec elk tail -n 100 /var/log/kibana/kibana5.log and get this log https://gist.github.com/anggara-ajip/4b02892ced02f3f9146d19b60140e67f what should I do next ?

anggara-ajip commented 4 years ago

@hacfi and for the record my .kibana_1 indices health status is RED now is that made this problem ?

anggara-ajip commented 4 years ago

this is my cluster health output

{ "cluster_name" : "elasticsearch", "status" : "red", "timed_out" : true, "number_of_nodes" : 3, "number_of_data_nodes" : 3, "active_primary_shards" : 21, "active_shards" : 39, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 5, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 88.63636363636364 }

hacfi commented 4 years ago

Looks like Kibana itself isn't the problem but rather Elasticsearch. Have you run out of disk space? Or is it a memory issue? In case of memory issue I'd restart the machine.

anggara-ajip commented 4 years ago

I still have 15% free disk space (69 GB free space) and 14 GB unallocated memory and I try create new elk container using same image and it works well

I already restart the machine several time

anggara-ajip commented 4 years ago

@hacfi is there any suggestion for me sir ?

hacfi commented 4 years ago

No idea..you can have a look at https://www.elastic.co/blog/red-elasticsearch-cluster-panic-no-longer but I'm not an expert at debugging Elasticsearch runtime issues.

anggara-ajip commented 4 years ago

thank you

anggara-ajip commented 4 years ago

still doesn't have the answer

spujadas commented 4 years ago

Closing as not an issue with the image itself.

"status" : "red",

For help with Elasticsearch runtime issues, please seek assistance from support forums such as https://discuss.elastic.co/