search

telekom-security / tpotce

🍯 T-Pot - The All In One Multi Honeypot Platform 🐝
GNU General Public License v3.0
6.77k stars 1.07k forks source link

Kibana service is not active. #445

Closed alancdias7 closed 5 years ago

alancdias7 commented 5 years ago

Good afternoon, I successfully installed T-POT 19.03 and it worked for a few weeks, but at this time the kibana service does not remain in active status. Has anyone been through this and / or has any north to give me?

Thankful.

Below the logs.

ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["license","info","xpack"],"pid":1,"message":"Imported licens
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:xpack_main@6.7.2","info"],"pid":1,"state":"
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:graph@6.7.2","info"],"pid":1,"state":"green
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:searchprofiler@6.7.2","info"],"pid":1,"stat
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:ml@6.7.2","info"],"pid":1,"state":"green","
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:tilemap@6.7.2","info"],"pid":1,"state":"gre
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:watcher@6.7.2","info"],"pid":1,"state":"gre
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:grokdebugger@6.7.2","info"],"pid":1,"state"
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:beats_management@6.7.2","info"],"pid":1,"st
ago 26 19:42:24 boringhumidity docker-compose[1089]: kibana                 | {"type":"log","@timestamp":"2019-08-26T19:23:57Z","tags":["status","plugin:index_management@6.7.2","info"],"pid":1,"st

{"type":"log","@timestamp":"2019-08-26T19:52:14Z","tags":["status","plugin:index_management@6.7.2","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-08-26T19:52:14Z","tags":["status","plugin:console@6.7.2","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-08-26T19:52:14Z","tags":["status","plugin:console_extensions@6.7.2","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-08-26T19:52:14Z","tags":["status","plugin:notifications@6.7.2","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-08-26T19:52:14Z","tags":["status","plugin:index_lifecycle_management@6.7.2","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-08-26T19:52:14Z","tags":["status","plugin:rollup@6.7.2","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-08-26T19:52:15Z","tags":["status","plugin:remote_clusters@6.7.2","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-08-26T19:52:15Z","tags":["status","plugin:cross_cluster_replication@6.7.2","info"],"pid":1,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2019-08-26T19:52:15Z","tags":["status","plugin:translations@6.7.2","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}

image

t3chn0m4g3 commented 5 years ago

Based on the logs I cannot tell. Run a docker logs kibana --follow and check post some screenshots of Head. Also give more context, since this is not an install from an ISO.

alancdias7 commented 5 years ago

Thank you for your support.

The installation was performed following the steps of the link: https://dtag-dev-sec.github.io/mediator/feature/2019/04/01/tpot-1903.html

git clone https://github.com/dtag-dev-sec/tpotce cd tpotce / iso / installer / ./install.sh --type = user

After the problem with kibana pending updates have been installed, but the error remains. This behavior started suddenly, without any manual interaction in TPOT services.

Below is the result of docker logs kibana --follow.

image image image image image image image

t3chn0m4g3 commented 5 years ago

The index has a problem. Please open ES Head and check for yellow / red flagged indices. This usually happens if the machine was not cleanly shutdown. However a reinstallation is usually faster.

alancdias7 commented 5 years ago

Thanks for the quick response, could you better guide me on these steps?

They are a layman on Linux.

alancdias7 commented 5 years ago

Would it be this?

image

t3chn0m4g3 commented 5 years ago

Look at the top navigation bar and click ES Head

alancdias7 commented 5 years ago

image

image

t3chn0m4g3 commented 5 years ago

Scroll to the right and check for yellow or red.

alancdias7 commented 5 years ago

Kibana is yellow.

image

alancdias7 commented 5 years ago

{ "uuid": "cI8QFQ4eSImnFz22l2kDEA", "primaries": { "docs": { "count": 311, "deleted": 1 }, "store": { "size_in_bytes": 346446 }, "indexing": { "index_total": 0, "index_time_in_millis": 0, "index_current": 0, "index_failed": 0, "delete_total": 0, "delete_time_in_millis": 0, "delete_current": 0, "noop_update_total": 0, "is_throttled": false, "throttle_time_in_millis": 0 }, "get": { "total": 4, "time_in_millis": 650, "exists_total": 0, "exists_time_in_millis": 0, "missing_total": 4, "missing_time_in_millis": 650, "current": 0 }, "search": { "open_contexts": 0, "query_total": 12, "query_time_in_millis": 2006, "query_current": 0, "fetch_total": 12, "fetch_time_in_millis": 705, "fetch_current": 0, "scroll_total": 0, "scroll_time_in_millis": 0, "scroll_current": 0, "suggest_total": 0, "suggest_time_in_millis": 0, "suggest_current": 0 }, "merges": { "current": 0, "current_docs": 0, "current_size_in_bytes": 0, "total": 0, "total_time_in_millis": 0, "total_docs": 0, "total_size_in_bytes": 0, "total_stopped_time_in_millis": 0, "total_throttled_time_in_millis": 0, "total_auto_throttle_in_bytes": 20971520 }, "refresh": { "total": 2, "total_time_in_millis": 0, "listeners": 0 }, "flush": { "total": 0, "periodic": 0, "total_time_in_millis": 0 }, "warmer": { "current": 0, "total": 1, "total_time_in_millis": 0 }, "query_cache": { "memory_size_in_bytes": 0, "total_count": 0, "hit_count": 0, "miss_count": 0, "cache_size": 0, "cache_count": 0, "evictions": 0 }, "fielddata": { "memory_size_in_bytes": 424, "evictions": 0 }, "completion": { "size_in_bytes": 0 }, "segments": { "count": 2, "memory_in_bytes": 16076, "terms_memory_in_bytes": 12496, "stored_fields_memory_in_bytes": 688, "term_vectors_memory_in_bytes": 0, "norms_memory_in_bytes": 2112, "points_memory_in_bytes": 44, "doc_values_memory_in_bytes": 736, "index_writer_memory_in_bytes": 0, "version_map_memory_in_bytes": 0, "fixed_bit_set_memory_in_bytes": 0, "max_unsafe_auto_id_timestamp": -1, "file_sizes": { } }, "translog": { "operations": 0, "size_in_bytes": 1815, "uncommitted_operations": 0, "uncommitted_size_in_bytes": 1815, "earliest_last_modified_age": 0 }, "request_cache": { "memory_size_in_bytes": 1768, "evictions": 0, "hit_count": 2, "miss_count": 2 }, "recovery": { "current_as_source": 0, "current_as_target": 0, "throttle_time_in_millis": 0 } }, "total": { "docs": { "count": 311, "deleted": 1 }, "store": { "size_in_bytes": 346446 }, "indexing": { "index_total": 0, "index_time_in_millis": 0, "index_current": 0, "index_failed": 0, "delete_total": 0, "delete_time_in_millis": 0, "delete_current": 0, "noop_update_total": 0, "is_throttled": false, "throttle_time_in_millis": 0 }, "get": { "total": 4, "time_in_millis": 650, "exists_total": 0, "exists_time_in_millis": 0, "missing_total": 4, "missing_time_in_millis": 650, "current": 0 }, "search": { "open_contexts": 0, "query_total": 12, "query_time_in_millis": 2006, "query_current": 0, "fetch_total": 12, "fetch_time_in_millis": 705, "fetch_current": 0, "scroll_total": 0, "scroll_time_in_millis": 0, "scroll_current": 0, "suggest_total": 0, "suggest_time_in_millis": 0, "suggest_current": 0 }, "merges": { "current": 0, "current_docs": 0, "current_size_in_bytes": 0, "total": 0, "total_time_in_millis": 0, "total_docs": 0, "total_size_in_bytes": 0, "total_stopped_time_in_millis": 0, "total_throttled_time_in_millis": 0, "total_auto_throttle_in_bytes": 20971520 }, "refresh": { "total": 2, "total_time_in_millis": 0, "listeners": 0 }, "flush": { "total": 0, "periodic": 0, "total_time_in_millis": 0 }, "warmer": { "current": 0, "total": 1, "total_time_in_millis": 0 }, "query_cache": { "memory_size_in_bytes": 0, "total_count": 0, "hit_count": 0, "miss_count": 0, "cache_size": 0, "cache_count": 0, "evictions": 0 }, "fielddata": { "memory_size_in_bytes": 424, "evictions": 0 }, "completion": { "size_in_bytes": 0 }, "segments": { "count": 2, "memory_in_bytes": 16076, "terms_memory_in_bytes": 12496, "stored_fields_memory_in_bytes": 688, "term_vectors_memory_in_bytes": 0, "norms_memory_in_bytes": 2112, "points_memory_in_bytes": 44, "doc_values_memory_in_bytes": 736, "index_writer_memory_in_bytes": 0, "version_map_memory_in_bytes": 0, "fixed_bit_set_memory_in_bytes": 0, "max_unsafe_auto_id_timestamp": -1, "file_sizes": { } }, "translog": { "operations": 0, "size_in_bytes": 1815, "uncommitted_operations": 0, "uncommitted_size_in_bytes": 1815, "earliest_last_modified_age": 0 }, "request_cache": { "memory_size_in_bytes": 1768, "evictions": 0, "hit_count": 2, "miss_count": 2 }, "recovery": { "current_as_source": 0, "current_as_target": 0, "throttle_time_in_millis": 0 } } }

t3chn0m4g3 commented 5 years ago

No, that's just a link. Here is what you can do:

  1. in ES head delete the kibana indices
  2. open a shell and run 
    sudo su -
systemctl stop tpot
tar xvfz /opt/tpot/etc/objects/elkbase.tgz -C /
reboot

If that does not work, please reinstall.

alancdias7 commented 5 years ago

Thank you so much for your time and help. Your recommendations solved the problem.

image

image

t3chn0m4g3 commented 5 years ago

Perfect, thanks for the feedback.