HELK never finishes install

syloktools commented 5 years ago

Ubuntu 18 Latest pull

The HELK gets to here and just hangs, been there for a couple hours:

I cannot hit the webpage and I get prompted for a username and password and then bad gateway.

Ol1ver26 commented 5 years ago

i have the same issue as well

Ol1ver26 commented 5 years ago

after I found that option 4 was not installed, I chose option 2, and this is my out-put for helk-elastalert and docker ps.

BTW, can i uninstall option 2 and install option after you figure out what goes wrong?

neu5ron commented 5 years ago

yup, you can choose a new install option after we figure out whats going on.

even with this option “2”. did the install script hang & you have a bad gateway (after user/password prompt) like previously mentioned?

Give me a little bit (an hour or so) and I will test a new install to replicate whats going on. hopefully this is just bad luck, right as we are in the process of pushing a bunch of fixes.

Ol1ver26 commented 5 years ago

Thanks for reply so fast, When I installed option 2, the entire installation process was completed in a very short time. no hang or other things happen.

neu5ron commented 5 years ago

I just ran option 4 with no issues, granted it took a little bit of time.. did you use the trail or basic license?

are you able to send the output of the following file, after you run the install with option 4? cat /var/log/helk-install.log

Ol1ver26 commented 5 years ago

i was use basic license

for install option4, do i have to uninstall option2 first then install option4? or its other way to do?

Ol1ver26 commented 5 years ago

and if i do need to uninstall option3 first, what code should i use to unstiall them sorry for asking so many questiones, i am new to this area

neu5ron commented 5 years ago

no problem. You do not need to uninstall. However, if you hit any errors about not having enough memory then run “sudo ./helk_remove_containers.sh” before new install.

On Sun, Sep 1, 2019 at 6:55 AM Ol1ver26 notifications@github.com wrote:

and if i do need to uninstall option3 first, what code should i use to unstiall them sorry for asking so many questiones, i am new to this area

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Cyb3rWard0g/HELK/issues/321?email_source=notifications&email_token=ABQOXYWTLW2QRQ7K7ORG7TDQHONQJA5CNFSM4ISSF3TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5T7WBQ#issuecomment-526908166, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQOXYQUYPUCH4HITQSKDCLQHONQJANCNFSM4ISSF3TA .

Ol1ver26 commented 5 years ago

I got this error

neu5ron commented 5 years ago

go ahead and ignore that, try the install now

On Mon, Sep 2, 2019 at 6:07 AM Ol1ver26 notifications@github.com wrote:

I got this error

[image: image] https://user-images.githubusercontent.com/36188251/64107178-3c8e9100-cdbd-11e9-935b-107750f9455f.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Cyb3rWard0g/HELK/issues/321?email_source=notifications&email_token=ABQOXYQWTJXIVHPOOCBSF23QHTQVVA5CNFSM4ISSF3TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5VL46Y#issuecomment-527089275, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQOXYSJ3DYA4UOJOOXWBX3QHTQVVANCNFSM4ISSF3TA .

syloktools commented 5 years ago

I will reinstall now and let you know as well.

syloktools commented 5 years ago

I am installing now, I noticed this log a couple times during the install:


update-alternatives: using /usr/share/postgresql/10/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.

neu5ron commented 5 years ago

hmmm, what operating system is this? and version? example: “Ubuntu 18.04” or “Centos 7”

On Mon, Sep 2, 2019 at 10:39 AM Robert Nixon notifications@github.com wrote:

I am installing now, I noticed this log a couple times during the install:

update-alternatives: using /usr/share/postgresql/10/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode invoke-rc.d: could not determine current runlevel invoke-rc.d: policy-rc.d denied execution of start.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Cyb3rWard0g/HELK/issues/321?email_source=notifications&email_token=ABQOXYXB7XZ4N64ED5VKTITQHUQTHA5CNFSM4ISSF3TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5V722Q#issuecomment-527170922, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQOXYWXGBW4LF2WU4R7MFDQHUQTHANCNFSM4ISSF3TA .

Ol1ver26 commented 5 years ago

go ahead and ignore that, try the install now … On Mon, Sep 2, 2019 at 6:07 AM Ol1ver26 @.***> wrote: I got this error [image: image] https://user-images.githubusercontent.com/36188251/64107178-3c8e9100-cdbd-11e9-935b-107750f9455f.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#321?email_source=notifications&email_token=ABQOXYQWTJXIVHPOOCBSF23QHTQVVA5CNFSM4ISSF3TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5VL46Y#issuecomment-527089275>, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQOXYSJ3DYA4UOJOOXWBX3QHTQVVANCNFSM4ISSF3TA .

i did, but helk_install.sh doesnt allow me, says i dont have enough memory, i have 16 GB ram (67% used by option2) and 150GB disk space

syloktools commented 5 years ago

@neu5ron This is Ubuntu 18.04

neu5ron commented 5 years ago

@Ol1ver26 @robertnixon2003 go ahead and run these commands in order, skip over if any errors and then try the install.. we need to update the logic for the remove script some more..

sudo docker stop $(docker ps --format '{{.Names}}' | grep -E '^helk\-')
sudo docker rm $(docker ps -a --format '{{.Names}}' | grep -E '^helk\-')
sudo docker rmi $(docker images -a | awk '{ print $1,$3 }' | grep 'cyb3rward0g\|helk' | awk '{print $2}')
sudo docker rmi $(docker images -a | awk '{ print $1,$3 }' | grep cp-ksql | awk '{print $2}')
sudo docker rmi $(docker images -a | awk '{ print $1,$3 }' | grep 'logstash\|kibana\|elasticsearch' | awk '{print $2}')

syloktools commented 5 years ago

Got this error:

root@tia-repo:/opt/HELK/docker# sudo docker rmi $(docker images -a | awk '{ print $1,$3 }' | grep 'cyb3rward0g\|helk' | awk '{print $2}')
Error response from daemon: conflict: unable to delete 6564c2c4654e (cannot be forced) - image has dependent child images

syloktools commented 5 years ago

I had to run this as well to get rid of the error above:

docker rmi $(docker images -q) -f

syloktools commented 5 years ago

So after doing that and trying a reinstall it still hangs at waiting for services to come up. That was 2 hours ago when I started. Going to try option 3.

neu5ron commented 5 years ago

sorry for all these troubles Can you send the output of /var/log/helk-install.log

On Mon, Sep 2, 2019 at 6:14 PM Robert Nixon notifications@github.com wrote:

So after doing that and trying a reinstall it still hangs at waiting for services to come up. That was 2 hours ago when I started. Going to try option 3.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cyb3rWard0g/HELK/issues/321?email_source=notifications&email_token=ABQOXYQ2RESPUJ3YRRJTAWTQHWF27A5CNFSM4ISSF3TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5WTK3Y#issuecomment-527250799, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQOXYXSI3EXGBV7JJHYUOTQHWF27ANCNFSM4ISSF3TA .

Cyb3rWard0g commented 5 years ago

Good evening @robertnixon2003 , I cannot replicate your issues with option 4. A few things before we keep re-installing and removing containers.

1) Please use a fresh Ubuntu (16 or 18). 2) Run option 4. I did that on my fresh Ubuntu 18 Bionic:

ubuntu@ip-10-0-0-97:~/HELK/docker$ sudo ./helk_install.sh 

**********************************************
**          HELK - THE HUNTING ELK          **
**                                          **
** Author: Roberto Rodriguez (@Cyb3rWard0g) **
** HELK build version: v0.1.8-alpha05292019 **
** HELK ELK version: 7.1.0                  **
** License: GPL-3.0                         **
**********************************************

[HELK-INSTALLATION-INFO] HELK being hosted on a Linux box
[HELK-INSTALLATION-INFO] Available Memory: 15623 MBs
[HELK-INSTALLATION-INFO] You're using ubuntu version bionic

*****************************************************
*      HELK - Docker Compose Build Choices          *
*****************************************************

1. KAFKA + KSQL + ELK + NGNIX
2. KAFKA + KSQL + ELK + NGNIX + ELASTALERT
3. KAFKA + KSQL + ELK + NGNIX + SPARK + JUPYTER
4. KAFKA + KSQL + ELK + NGNIX + SPARK + JUPYTER + ELASTALERT

Enter build choice [ 1 - 4]: 4
[HELK-INSTALLATION-INFO] HELK build set to 4
[HELK-INSTALLATION-INFO] Set HELK elastic subscription (basic or trial): basic
[HELK-INSTALLATION-INFO] Set HELK IP. Default value is your current IP: 10.0.0.97
[HELK-INSTALLATION-INFO] Set HELK Kibana UI Password: hunting
[HELK-INSTALLATION-INFO] Verify HELK Kibana UI Password: hunting
[HELK-INSTALLATION-INFO] Installing htpasswd..
[HELK-INSTALLATION-INFO] Installing docker via convenience script..
[HELK-INSTALLATION-INFO] Installing docker-compose..
[HELK-INSTALLATION-INFO] Checking local vm.max_map_count variable and setting it to 4120294
[HELK-INSTALLATION-INFO] Building & running HELK from helk-kibana-notebook-analysis-alert-basic.yml file..
[HELK-INSTALLATION-INFO] Waiting for some services to be up .....

***********************************************************************************
** [HELK-INSTALLATION-INFO] HELK WAS INSTALLED SUCCESSFULLY                      **
** [HELK-INSTALLATION-INFO] USE THE FOLLOWING SETTINGS TO INTERACT WITH THE HELK **
***********************************************************************************

HELK KIBANA URL: https://10.0.0.97
HELK KIBANA USER: helk
HELK KIBANA PASSWORD: hunting
HELK SPARK MASTER UI: http://10.0.0.97:8080
HELK JUPYTER SERVER URL: http://10.0.0.97/jupyter
HELK JUPYTER CURRENT TOKEN: 7a95abe70e0f8470c26a87c53a41d374635d693d3f1a355e
HELK ZOOKEEPER: 10.0.0.97:2181
HELK KSQL SERVER: 10.0.0.97:8088

IT IS HUNTING SEASON!!!!!

This is my helk-install-log file:

ubuntu@ip-10-0-0-97:~$ tail -f /var/log/helk-install.log 
100 15.4M  100 15.4M    0     0  52.6M      0 --:--:-- --:--:-- --:--:-- 52.6M
Creating network "docker_helk" with driver "bridge"
Creating volume "docker_esdata" with local driver
Creating volume "docker_notebooks" with local driver
Pulling helk-elasticsearch (docker.elastic.co/elasticsearch/elasticsearch:7.1.0)...
7.1.0: Pulling from elasticsearch/elasticsearch
Digest: sha256:802b6a299260dbaf21a9c57e3a634491ff788a1ea13a51598d4cd105739509c4
Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.1.0
Pulling helk-kibana (docker.elastic.co/kibana/kibana:7.1.0)...
7.1.0: Pulling from kibana/kibana
Digest: sha256:0eb53e2eb9a8846a5a4e700115a4b6e207a8f298128a45cdd28a514f708ec0f3
Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.1.0
Pulling helk-logstash (docker.elastic.co/logstash/logstash:7.1.0)...
7.1.0: Pulling from logstash/logstash
Digest: sha256:9258bd2ef10c084a267e470e4fac7b5144e2bd932d628deab4f2c8cc2ff47dd0
Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.1.0
Pulling helk-nginx (cyb3rward0g/helk-nginx:0.0.7)...
0.0.7: Pulling from cyb3rward0g/helk-nginx
Digest: sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0
Status: Downloaded newer image for cyb3rward0g/helk-nginx:0.0.7
Pulling helk-zookeeper (cyb3rward0g/helk-zookeeper:2.2.0)...
2.2.0: Pulling from cyb3rward0g/helk-zookeeper
Digest: sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b
Status: Downloaded newer image for cyb3rward0g/helk-zookeeper:2.2.0
Pulling helk-kafka-broker (cyb3rward0g/helk-kafka-broker:2.2.0)...
2.2.0: Pulling from cyb3rward0g/helk-kafka-broker
Digest: sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903
Status: Downloaded newer image for cyb3rward0g/helk-kafka-broker:2.2.0
Pulling helk-ksql-server (confluentinc/cp-ksql-server:5.1.3)...
5.1.3: Pulling from confluentinc/cp-ksql-server
Digest: sha256:063add111cc93b1a0118f88b577e31303045d4cc08eb1d21458429f05cba4b02
Status: Downloaded newer image for confluentinc/cp-ksql-server:5.1.3
Pulling helk-ksql-cli (confluentinc/cp-ksql-cli:5.1.3)...
5.1.3: Pulling from confluentinc/cp-ksql-cli
Digest: sha256:18c0ccb00fbf87679e16e9e0da600548fcb236a2fd173263b09e89b2d3a42cc3
Status: Downloaded newer image for confluentinc/cp-ksql-cli:5.1.3
Building helk-jupyter
Step 1/16 : FROM cyb3rward0g/jupyter-hunt:0.0.4
0.0.4: Pulling from cyb3rward0g/jupyter-hunt
Digest: sha256:f788e827c295502f29142a358338149c937442181670982f96a94a793cc8d9e2
Status: Downloaded newer image for cyb3rward0g/jupyter-hunt:0.0.4
 ---> 6564c2c4654e
Step 2/16 : LABEL maintainer="Roberto Rodriguez @Cyb3rWard0g"
 ---> Running in f61e158de0f0
Removing intermediate container f61e158de0f0
 ---> 05c8f7d9204f
Step 3/16 : LABEL description="Dockerfile Notebooks-Forge Jupyter-Hunt Project."
 ---> Running in 8a69ef4720dc
Removing intermediate container 8a69ef4720dc
 ---> 25a2a94a5159
Step 4/16 : ENV DEBIAN_FRONTEND noninteractive
 ---> Running in da18859c43d9
Removing intermediate container da18859c43d9
 ---> b4acac8aba03
Step 5/16 : USER root
 ---> Running in e277c96b7160
Removing intermediate container e277c96b7160
 ---> 4a28c1881d5b
Step 6/16 : ENV POSTGRESQL_VERSION=42.2.5
 ---> Running in 6ed8c58c0738
Removing intermediate container 6ed8c58c0738
 ---> 5cd2cc17515f
Step 7/16 : RUN mkdir /opt/jupyter/notebooks/datasets     && apt-get update --fix-missing && apt-get install -y --no-install-recommends     postgresql postgresql-contrib     && apt-get -qy clean autoremove     && rm -rf /var/lib/apt/lists/*     && wget https://jdbc.postgresql.org/download/postgresql-${POSTGRESQL_VERSION}.jar -P /opt/jupyter/spark/jars/
 ---> Running in fa4365bbeb46
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 http://security.ubuntu.com/ubuntu bionic-security/restricted Sources [1976 B]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:5 http://security.ubuntu.com/ubuntu bionic-security/universe Sources [200 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:7 http://security.ubuntu.com/ubuntu bionic-security/main Sources [157 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/universe Sources [11.5 MB]
Get:9 http://security.ubuntu.com/ubuntu bionic-security/multiverse Sources [2773 B]
Get:10 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [628 kB]
Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [6222 B]
Get:12 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [4173 B]
Get:13 http://archive.ubuntu.com/ubuntu bionic/multiverse Sources [216 kB]
Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [760 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic/main Sources [1063 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic/restricted Sources [5823 B]
Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1344 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:19 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:20 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Sources [5282 B]
Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Sources [343 kB]
Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main Sources [373 kB]
Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Sources [4184 B]
Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [938 kB]
Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [18.5 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1283 kB]
Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [8000 B]
Get:29 http://archive.ubuntu.com/ubuntu bionic-backports/universe Sources [2516 B]
Get:30 http://archive.ubuntu.com/ubuntu bionic-backports/main Sources [2569 B]
Get:31 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [2496 B]
Get:32 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [4212 B]
Fetched 30.9 MB in 10s (3193 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libicu60 libpq5 libxml2 libxslt1.1 netbase postgresql-10
  postgresql-client-10 postgresql-client-common postgresql-common ssl-cert
Suggested packages:
  postgresql-doc locales-all postgresql-doc-10 libjson-perl openssl-blacklist
Recommended packages:
  sysstat
The following NEW packages will be installed:
  libicu60 libpq5 libxml2 libxslt1.1 netbase postgresql postgresql-10
  postgresql-client-10 postgresql-client-common postgresql-common
  postgresql-contrib ssl-cert
0 upgraded, 12 newly installed, 0 to remove and 83 not upgraded.
Need to get 13.9 MB of archives.
After this operation, 54.1 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 libicu60 amd64 60.2-3ubuntu3 [8054 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxml2 amd64 2.9.4+dfsg1-6.1ubuntu1.2 [663 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 netbase all 5.4 [12.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpq5 amd64 10.10-0ubuntu0.18.04.1 [108 kB]
Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.1 [150 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 postgresql-client-common all 190 [29.5 kB]
Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 postgresql-client-10 amd64 10.10-0ubuntu0.18.04.1 [935 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 ssl-cert all 1.0.39 [17.0 kB]
Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 postgresql-common all 190 [157 kB]
Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 postgresql-10 amd64 10.10-0ubuntu0.18.04.1 [3758 kB]
Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 postgresql all 10+190 [5784 B]
Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 postgresql-contrib all 10+190 [5796 B]
Preconfiguring packages ...
Fetched 13.9 MB in 5s (2788 kB/s)
Selecting previously unselected package libicu60:amd64.
(Reading database ... 20243 files and directories currently installed.)
Preparing to unpack .../00-libicu60_60.2-3ubuntu3_amd64.deb ...
Unpacking libicu60:amd64 (60.2-3ubuntu3) ...
Selecting previously unselected package libxml2:amd64.
Preparing to unpack .../01-libxml2_2.9.4+dfsg1-6.1ubuntu1.2_amd64.deb ...
Unpacking libxml2:amd64 (2.9.4+dfsg1-6.1ubuntu1.2) ...
Selecting previously unselected package netbase.
Preparing to unpack .../02-netbase_5.4_all.deb ...
Unpacking netbase (5.4) ...
Selecting previously unselected package libpq5:amd64.
Preparing to unpack .../03-libpq5_10.10-0ubuntu0.18.04.1_amd64.deb ...
Unpacking libpq5:amd64 (10.10-0ubuntu0.18.04.1) ...
Selecting previously unselected package libxslt1.1:amd64.
Preparing to unpack .../04-libxslt1.1_1.1.29-5ubuntu0.1_amd64.deb ...
Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.1) ...
Selecting previously unselected package postgresql-client-common.
Preparing to unpack .../05-postgresql-client-common_190_all.deb ...
Unpacking postgresql-client-common (190) ...
Selecting previously unselected package postgresql-client-10.
Preparing to unpack .../06-postgresql-client-10_10.10-0ubuntu0.18.04.1_amd64.deb ...
Unpacking postgresql-client-10 (10.10-0ubuntu0.18.04.1) ...
Selecting previously unselected package ssl-cert.
Preparing to unpack .../07-ssl-cert_1.0.39_all.deb ...
Unpacking ssl-cert (1.0.39) ...
Selecting previously unselected package postgresql-common.
Preparing to unpack .../08-postgresql-common_190_all.deb ...
Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common'
Unpacking postgresql-common (190) ...
Selecting previously unselected package postgresql-10.
Preparing to unpack .../09-postgresql-10_10.10-0ubuntu0.18.04.1_amd64.deb ...
Unpacking postgresql-10 (10.10-0ubuntu0.18.04.1) ...
Selecting previously unselected package postgresql.
Preparing to unpack .../10-postgresql_10+190_all.deb ...
Unpacking postgresql (10+190) ...
Selecting previously unselected package postgresql-contrib.
Preparing to unpack .../11-postgresql-contrib_10+190_all.deb ...
Unpacking postgresql-contrib (10+190) ...
Setting up libicu60:amd64 (60.2-3ubuntu3) ...
Setting up ssl-cert (1.0.39) ...
Setting up libxml2:amd64 (2.9.4+dfsg1-6.1ubuntu1.2) ...
Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.1) ...
Setting up libpq5:amd64 (10.10-0ubuntu0.18.04.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Setting up netbase (5.4) ...
Setting up postgresql-client-common (190) ...
Setting up postgresql-common (190) ...
Adding user postgres to group ssl-cert

Creating config file /etc/postgresql-common/createcluster.conf with new version
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up postgresql-client-10 (10.10-0ubuntu0.18.04.1) ...
update-alternatives: using /usr/share/postgresql/10/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode
Setting up postgresql-10 (10.10-0ubuntu0.18.04.1) ...
Creating new PostgreSQL cluster 10/main ...
/usr/lib/postgresql/10/bin/initdb -D /var/lib/postgresql/10/main --auth-local peer --auth-host md5
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/10/main ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    /usr/lib/postgresql/10/bin/pg_ctl -D /var/lib/postgresql/10/main -l logfile start

Ver Cluster Port Status Owner    Data directory              Log file
10  main    5432 down   postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log
update-alternatives: using /usr/share/postgresql/10/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up postgresql (10+190) ...
Setting up postgresql-contrib (10+190) ...
--2019-09-03 02:17:03--  https://jdbc.postgresql.org/download/postgresql-42.2.5.jar
Resolving jdbc.postgresql.org (jdbc.postgresql.org)... 72.32.157.228, 2001:4800:3e1:1::228
Connecting to jdbc.postgresql.org (jdbc.postgresql.org)|72.32.157.228|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 825943 (807K) [application/java-archive]
Saving to: ‘/opt/jupyter/spark/jars/postgresql-42.2.5.jar’

     0K .......... .......... .......... .......... ..........  6%  872K 1s
    50K .......... .......... .......... .......... .......... 12% 1.69M 1s
   100K .......... .......... .......... .......... .......... 18% 88.0M 0s
   150K .......... .......... .......... .......... .......... 24% 1.73M 0s
   200K .......... .......... .......... .......... .......... 30% 64.6M 0s
   250K .......... .......... .......... .......... .......... 37%  121M 0s
   300K .......... .......... .......... .......... .......... 43% 1.78M 0s
   350K .......... .......... .......... .......... .......... 49% 78.2M 0s
   400K .......... .......... .......... .......... .......... 55% 70.0M 0s
   450K .......... .......... .......... .......... .......... 61% 83.0M 0s
   500K .......... .......... .......... .......... .......... 68% 87.1M 0s
   550K .......... .......... .......... .......... .......... 74% 97.6M 0s
   600K .......... .......... .......... .......... .......... 80%  153M 0s
   650K .......... .......... .......... .......... .......... 86% 1.93M 0s
   700K .......... .......... .......... .......... .......... 92% 93.4M 0s
   750K .......... .......... .......... .......... .......... 99%  121M 0s
   800K ......                                                100%  121M=0.2s

2019-09-03 02:17:03 (4.55 MB/s) - ‘/opt/jupyter/spark/jars/postgresql-42.2.5.jar’ saved [825943/825943]

Removing intermediate container fa4365bbeb46
 ---> 158cd32a5e37
Step 8/16 : COPY notebooks/* ${JUPYTER_DIR}/notebooks/
 ---> 7638a7dcec52
Step 9/16 : COPY datasets/* ${JUPYTER_DIR}/notebooks/datasets/
 ---> cba32a269207
Step 10/16 : COPY spark/* ${SPARK_HOME}/conf/
 ---> 4b775cca68c9
Step 11/16 : COPY scripts/* ${JUPYTER_DIR}/scripts/
 ---> c421b69938a0
Step 12/16 : RUN chown -R ${USER} ${JUPYTER_DIR} ${HOME} ${SPARK_HOME}     && chown ${USER} /run/postgresql
 ---> Running in 8f976ac1c2af
Removing intermediate container 8f976ac1c2af
 ---> ab0f7face9ae
Step 13/16 : WORKDIR ${HOME}
 ---> Running in 3be3bf300f56
Removing intermediate container 3be3bf300f56
 ---> d4b730c1e97f
Step 14/16 : ENTRYPOINT ["/opt/jupyter/scripts/jupyter-entrypoint.sh"]
 ---> Running in dc41f01200dc
Removing intermediate container dc41f01200dc
 ---> 025684d6f19d
Step 15/16 : CMD ["/opt/jupyter/scripts/jupyter-cmd.sh"]
 ---> Running in e076300b52a0
Removing intermediate container e076300b52a0
 ---> dfb1bace2fea
Step 16/16 : USER ${USER}
 ---> Running in 7e0b4d7a7497
Removing intermediate container 7e0b4d7a7497
 ---> 4221c8a41ee5

Successfully built 4221c8a41ee5
Successfully tagged docker_helk-jupyter:latest
Pulling helk-spark-master (cyb3rward0g/helk-spark-master:2.4.3)...
2.4.3: Pulling from cyb3rward0g/helk-spark-master
Digest: sha256:987fe26268f431295ecd5fb681a22dbf33d63e8b17f17efd648c685ca6368584
Status: Downloaded newer image for cyb3rward0g/helk-spark-master:2.4.3
Pulling helk-spark-worker (cyb3rward0g/helk-spark-worker:2.4.3)...
2.4.3: Pulling from cyb3rward0g/helk-spark-worker
Digest: sha256:f360e1e96878a4b648cc7434011663702af264da240187441fd0fee5d076e7dd
Status: Downloaded newer image for cyb3rward0g/helk-spark-worker:2.4.3
Pulling helk-elastalert (cyb3rward0g/helk-elastalert:0.2.4)...
0.2.4: Pulling from cyb3rward0g/helk-elastalert
Digest: sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d
Status: Downloaded newer image for cyb3rward0g/helk-elastalert:0.2.4
Creating helk-elasticsearch ... done
Creating helk-kibana        ... done
Creating helk-logstash      ... done
Creating helk-nginx         ... done
Creating helk-jupyter       ... done
Creating helk-zookeeper     ... done
Creating helk-spark-master  ... done
Creating helk-elastalert    ... done
Creating helk-spark-worker  ... done
Creating helk-kafka-broker  ... done
Creating helk-ksql-server   ... done
Creating helk-ksql-cli      ... done

These are the containers running:

ubuntu@ip-10-0-0-97:~$ sudo docker ps
CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                                                      NAMES
9feac9724b8c        confluentinc/cp-ksql-cli:5.1.3                        "/bin/sh"                10 minutes ago      Up 10 minutes                                                                  helk-ksql-cli
d4ed4992baa6        confluentinc/cp-ksql-server:5.1.3                     "/etc/confluent/dock…"   10 minutes ago      Up 10 minutes       0.0.0.0:8088->8088/tcp                                     helk-ksql-server
b179d3750d75        cyb3rward0g/helk-kafka-broker:2.2.0                   "./kafka-entrypoint.…"   10 minutes ago      Up 10 minutes       0.0.0.0:9092->9092/tcp                                     helk-kafka-broker
59720afc2e18        cyb3rward0g/helk-spark-worker:2.4.3                   "./spark-worker-entr…"   10 minutes ago      Up 10 minutes                                                                  helk-spark-worker
2633376df642        cyb3rward0g/helk-spark-master:2.4.3                   "./spark-master-entr…"   10 minutes ago      Up 10 minutes       7077/tcp, 0.0.0.0:8080->8080/tcp                           helk-spark-master
04e985a7b77a        cyb3rward0g/helk-elastalert:0.2.4                     "./elastalert-entryp…"   10 minutes ago      Up 10 minutes                                                                  helk-elastalert
825615cc73a4        docker_helk-jupyter                                   "/opt/jupyter/script…"   10 minutes ago      Up 10 minutes       8000/tcp, 8888/tcp                                         helk-jupyter
a76bf3d8656a        cyb3rward0g/helk-zookeeper:2.2.0                      "./zookeeper-entrypo…"   10 minutes ago      Up 10 minutes       2181/tcp, 2888/tcp, 3888/tcp                               helk-zookeeper
1aac34f46ab4        cyb3rward0g/helk-nginx:0.0.7                          "/opt/helk/scripts/n…"   10 minutes ago      Up 10 minutes       0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp                   helk-nginx
6e8f64465153        docker.elastic.co/logstash/logstash:7.1.0             "/usr/share/logstash…"   10 minutes ago      Up 10 minutes       0.0.0.0:5044->5044/tcp, 0.0.0.0:8531->8531/tcp, 9600/tcp   helk-logstash
1ad3ef5f8eec        docker.elastic.co/kibana/kibana:7.1.0                 "/usr/share/kibana/s…"   10 minutes ago      Up 10 minutes       5601/tcp                                                   helk-kibana
4276bf133740        docker.elastic.co/elasticsearch/elasticsearch:7.1.0   "/usr/share/elastics…"   10 minutes ago      Up 10 minutes       9200/tcp, 9300/tcp                                         helk-elasticsearch
ubuntu@ip-10-0-0-97:~$

These are the only images that you should have and their tags:

ubuntu@ip-10-0-0-97:~$ sudo docker images
REPOSITORY                                      TAG                 IMAGE ID            CREATED             SIZE
docker_helk-jupyter                             latest              4221c8a41ee5        11 minutes ago      5.05GB
cyb3rward0g/jupyter-hunt                        0.0.4               6564c2c4654e        2 weeks ago         4.6GB
cyb3rward0g/helk-elastalert                     0.2.4               c7d5d7eed99c        3 months ago        753MB
cyb3rward0g/helk-spark-worker                   2.4.3               13a6d0c1fca5        3 months ago        605MB
cyb3rward0g/helk-spark-master                   2.4.3               b993d31e00fd        3 months ago        605MB
docker.elastic.co/logstash/logstash             7.1.0               93ae8cd11560        3 months ago        847MB
docker.elastic.co/kibana/kibana                 7.1.0               714b175e84e8        3 months ago        745MB
docker.elastic.co/elasticsearch/elasticsearch   7.1.0               12ad640a1ec0        3 months ago        894MB
confluentinc/cp-ksql-server                     5.1.3               785f91d9f484        3 months ago        529MB
confluentinc/cp-ksql-cli                        5.1.3               5901cec02503        3 months ago        525MB
cyb3rward0g/helk-kafka-broker                   2.2.0               c2f7cd82ae1d        4 months ago        382MB
cyb3rward0g/helk-zookeeper                      2.2.0               fa0e4ceb4a83        4 months ago        382MB
cyb3rward0g/helk-nginx                          0.0.7               280d044b6719        12 months ago       329MB

I cannot replicate what you are experiencing at the moment.

Cyb3rWard0g commented 5 years ago

update-alternatives: using /usr/share/postgresql/10/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode invoke-rc.d: could not determine current runlevel invoke-rc.d: policy-rc.d denied execution of start.

Where did you get that?? ^^ Can you share more information about where you are seeing those error messages?? Just in case, postgresql is part of helk-jupyter containers. Please run the following command and share the output:

sudo docker logs helk-jupyter

Thank you for your patience @robertnixon2003

Cyb3rWard0g commented 5 years ago

update-alternatives: using /usr/share/postgresql/10/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode invoke-rc.d: could not determine current runlevel invoke-rc.d: policy-rc.d denied execution of start.

Where did you get that?? ^^ Can you share more information about where you are seeing those error messages?? Just in case, postgresql is part of helk-jupyter containers. Please run the following command and share the output:

sudo docker logs helk-jupyter

Thank you for your patience @robertnixon2003

Hey @robertnixon2003 , I see were you get the postgresql messages. Ignore those. As you can see in the Jupyter Logs, the server is started properly:

ubuntu@ip-10-0-0-97:~$ sudo docker logs --follow helk-jupyter
The files belonging to this database system will be owned by user "jupyter".
This user must also own the server process.

The database cluster will be initialized with locale "C.UTF-8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

creating directory /home/jupyter/srv/pgsql ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    /usr/lib/postgresql/10/bin/pg_ctl -D /home/jupyter/srv/pgsql -l logfile start

[NOTEBOOK-JUPYTER-DOCKER-INSTALLATION-INFO] The files in this database system will be owned by user jupyter..
pg_ctl: no server running
waiting for server to start.... done
server started

Also I checked if everything is running properly from a postgresql perspective in the Jupyter container and it is:

ubuntu@ip-10-0-0-97:~$ sudo docker exec -ti helk-jupyter bash
jupyter@2b17f0a614d3:~$ /usr/lib/postgresql/10/bin/pg_ctl -D /home/jupyter/srv/pgsql status
pg_ctl: server is running (PID: 23)

In other words ignore the following messages:

invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.

I still cannot replicate why it would hang for hours for you.

syloktools commented 5 years ago

I will wipe the server and try again. Not sure whats going on. I installed it before and it worked fine on this same box.

sugarp0pe commented 5 years ago

Hello, I have the same issue too Ubuntu 18.04.3 Docker 19.03.2 docker-compose v.1.24.1 Running option 2 helk_install.sh elasticsearch container always restarting every 30 seconds

docker logs elasticsearch:7.1.0

[HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:42:20,336+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sever--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,346+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,351+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,352+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,353+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,353+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-1373145707971171539, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,501+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,505+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,512+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,514+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,514+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,515+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,515+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,516+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,516+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,517+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,521+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,522+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,522+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,525+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,526+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,533+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,534+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,534+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,537+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,545+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,549+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,551+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,553+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,554+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,555+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:42:48,258+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:43:05,175+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-1373145707971171539/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-1373145707971171539/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] } [HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:43:16,586+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sever--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,597+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,601+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,602+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,603+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,604+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-2864629304196106528, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,701+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,701+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,702+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,703+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,704+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,705+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,706+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,708+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,708+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,711+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,711+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,714+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,715+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,726+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,726+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,727+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,727+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,729+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,730+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,730+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,731+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,732+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,733+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,734+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,736+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:43:24,169+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:43:36,319+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-2864629304196106528/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-2864629304196106528/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] } [HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:43:46,183+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sever--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,188+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,193+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,194+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,194+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,195+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-3670512058239285020, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,189+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,191+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,192+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,193+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,194+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,198+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,201+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,206+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,209+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,211+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,211+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,212+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,212+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,213+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,217+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,218+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,223+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,224+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,229+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,230+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,230+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,233+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:43:53,630+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:44:05,915+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-3670512058239285020/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-3670512058239285020/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] }

neu5ron commented 5 years ago

Can you send the output of

/var/log/helk-install.log

On Sat, Sep 7, 2019 at 9:13 AM sugarp0pe notifications@github.com wrote:

Hello, I have the same issue too Ubuntu 18.04.3 Docker 19.03.2 docker-compose v.1.24.1 Running option 2 helk_install.sh elasticsearch container always restarting every 30 seconds docker logs elasticsearch:7.1.0 HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:42:20,336+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sentinel--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,346+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,351+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,352+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,353+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,353+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-1373145707971171539, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,501+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,505+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,512+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,514+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,514+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,515+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,515+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,516+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,516+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,517+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,521+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,522+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,522+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,525+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,526+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,533+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,534+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,534+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,537+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,545+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,549+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,551+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,553+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,554+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,555+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:42:48,258+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:43:05,175+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-1373145707971171539/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-1373145707971171539/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] } [HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:43:16,586+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sentinel--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,597+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,601+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,602+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,603+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,604+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-2864629304196106528, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,701+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,701+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,702+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,703+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,704+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,705+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,706+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,708+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,708+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,711+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,711+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,714+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,715+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,726+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,726+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,727+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,727+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,729+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,730+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,730+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,731+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,732+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,733+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,734+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,736+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:43:24,169+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:43:36,319+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-2864629304196106528/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-2864629304196106528/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] } [HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:43:46,183+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sentinel--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,188+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,193+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,194+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,194+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,195+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-3670512058239285020, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,189+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,191+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,192+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,193+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,194+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,198+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,201+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,206+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,209+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,211+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,211+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,212+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,212+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,213+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,217+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,218+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,223+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,224+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,229+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,230+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,230+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", " node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,233+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:43:53,630+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:44:05,915+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-3670512058239285020/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-3670512058239285020/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cyb3rWard0g/HELK/issues/321?email_source=notifications&email_token=ABQOXYVKGF2KKIXUAE3BLC3QIOSFZA5CNFSM4ISSF3TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6EYR2I#issuecomment-529107177, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQOXYQDSVER4PNVUULE7STQIOSFZANCNFSM4ISSF3TA .

sugarp0pe commented 5 years ago

Thank you for quick response

helk-install.log

WARNING: apt does not have a stable CLI interface. Use with caution in scripts. Reading package lists... Building dependency tree... Reading state information... The following additional packages will be installed: libapr1 libaprutil1 The following NEW packages will be installed: apache2-utils libapr1 libaprutil1 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. Need to get 259 kB of archives. After this operation, 865 kB of additional disk space will be used. Get:1 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libapr1 amd64 1.6.3-2 [90.9 kB] Get:2 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libaprutil1 amd64 1.6.1-2 [84.4 kB] Get:3 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-utils amd64 2.4.29-1ubuntu4.10 [83.9 kB] Fetched 259 kB in 3s (103 kB/s) Selecting previously unselected package libapr1:amd64. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102895 files and directories currently installed.) Preparing to unpack .../libapr1_1.6.3-2_amd64.deb ... Unpacking libapr1:amd64 (1.6.3-2) ... Selecting previously unselected package libaprutil1:amd64. Preparing to unpack .../libaprutil1_1.6.1-2_amd64.deb ... Unpacking libaprutil1:amd64 (1.6.1-2) ... Selecting previously unselected package apache2-utils. Preparing to unpack .../apache2-utils_2.4.29-1ubuntu4.10_amd64.deb ... Unpacking apache2-utils (2.4.29-1ubuntu4.10) ... Setting up libapr1:amd64 (1.6.3-2) ... Setting up libaprutil1:amd64 (1.6.1-2) ... Setting up apache2-utils (2.4.29-1ubuntu4.10) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Processing triggers for libc-bin (2.27-3ubuntu1) ... Adding password for user helk Creating network "docker_helk" with driver "bridge" Creating volume "docker_esdata" with local driver Pulling helk-elasticsearch (docker.elastic.co/elasticsearch/elasticsearch:7.1.0)... 7.1.0: Pulling from elasticsearch/elasticsearch Digest: sha256:802b6a299260dbaf21a9c57e3a634491ff788a1ea13a51598d4cd105739509c4 Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.1.0 Pulling helk-kibana (docker.elastic.co/kibana/kibana:7.1.0)... 7.1.0: Pulling from kibana/kibana Digest: sha256:0eb53e2eb9a8846a5a4e700115a4b6e207a8f298128a45cdd28a514f708ec0f3 Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.1.0 Pulling helk-logstash (docker.elastic.co/logstash/logstash:7.1.0)... 7.1.0: Pulling from logstash/logstash Digest: sha256:9258bd2ef10c084a267e470e4fac7b5144e2bd932d628deab4f2c8cc2ff47dd0 Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.1.0 Pulling helk-nginx (cyb3rward0g/helk-nginx:0.0.7)... 0.0.7: Pulling from cyb3rward0g/helk-nginx Digest: sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Status: Downloaded newer image for cyb3rward0g/helk-nginx:0.0.7 Pulling helk-zookeeper (cyb3rward0g/helk-zookeeper:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-zookeeper Digest: sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Status: Downloaded newer image for cyb3rward0g/helk-zookeeper:2.2.0 Pulling helk-kafka-broker (cyb3rward0g/helk-kafka-broker:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-kafka-broker Digest: sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Status: Downloaded newer image for cyb3rward0g/helk-kafka-broker:2.2.0 Pulling helk-ksql-server (confluentinc/cp-ksql-server:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-server Digest: sha256:063add111cc93b1a0118f88b577e31303045d4cc08eb1d21458429f05cba4b02 Status: Downloaded newer image for confluentinc/cp-ksql-server:5.1.3 Pulling helk-ksql-cli (confluentinc/cp-ksql-cli:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-cli Digest: sha256:18c0ccb00fbf87679e16e9e0da600548fcb236a2fd173263b09e89b2d3a42cc3 Status: Downloaded newer image for confluentinc/cp-ksql-cli:5.1.3 Pulling helk-elastalert (cyb3rward0g/helk-elastalert:0.2.4)... 0.2.4: Pulling from cyb3rward0g/helk-elastalert Digest: sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Status: Downloaded newer image for cyb3rward0g/helk-elastalert:0.2.4 Creating helk-elasticsearch ... [1A[2K Creating helk-elasticsearch ... [32mdone[0m [1BCreating helk-kibana ... [1A[2K Creating helk-kibana ... [32mdone[0m [1BCreating helk-logstash ... Creating helk-nginx ... [2A[2K Creating helk-logstash ... [32mdone[0m [2BCreating helk-elastalert ... Creating helk-zookeeper ... [3A[2K Creating helk-nginx ... [32mdone[0m [3B[2A[2K Creating helk-elastalert ... [32mdone[0m [2B[1A[2K Creating helk-zookeeper ... [32mdone[0m [1BCreating helk-kafka-broker ... [1A[2K Creating helk-kafka-broker ... [32mdone[0m [1BCreating helk-ksql-server ... [1A[2K Creating helk-ksql-server ... [32mdone[0m [1BCreating helk-ksql-cli ... [1A[2K Creating helk-ksql-cli ... [32mdone[0m [1BAdding password for user helk helk-elasticsearch is up-to-date helk-kibana is up-to-date helk-logstash is up-to-date helk-nginx is up-to-date helk-elastalert is up-to-date helk-zookeeper is up-to-date helk-kafka-broker is up-to-date helk-ksql-server is up-to-date helk-ksql-cli is up-to-date 885d61b0f8a5 5b868662e349 a29fd6d11d79 a45a104eca2b 885d61b0f8a5 5b868662e349 a29fd6d11d79 a45a104eca2b Untagged: cyb3rward0g/helk-elastalert:0.2.4 Untagged: cyb3rward0g/helk-elastalert@sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Deleted: sha256:c7d5d7eed99cce73b7073770dc3c6ecc56a0a447305027ece73e4fabfa0ca51c Deleted: sha256:7ddfedd66acd6b1603c988457aa5cd33417affc4fd9d2e3d768e7ef327bd87e5 Deleted: sha256:19f62ea43ccbd84698f2a9375d08a20d52a02e7f4788843a4d5390cabd3c2472 Deleted: sha256:255396df8ad9053ed73d5a95c12d855e52ad967cd9054676efcfcd85310a1882 Deleted: sha256:df064f144aab0c435fcb883babd6597b724d8f2b73373976f4d3ed964fbbdcee Deleted: sha256:91fe17dde3260a0caf19608ce8e57a69ca0b16ac42e16d489cd08ddb48cb53e9 Deleted: sha256:6872aedee364733b71cd03b5e315ac483f9e6a0bfd61e9a15ca8c3fe7e539936 Untagged: cyb3rward0g/helk-kafka-broker:2.2.0 Untagged: cyb3rward0g/helk-kafka-broker@sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Deleted: sha256:c2f7cd82ae1da69dd48c2fc2c26ae855fc8c05efb9e7a8f5d985ebd6c99fddd7 Deleted: sha256:b9059b735d4f7ab0ab47f76fceee5c10d6674739f7d3b2cb89c6cf5db0f92293 Deleted: sha256:52b06347f9930845994d3a549cc9ff99524b3a6ce63ed35b85fa67aa7955ecf8 Deleted: sha256:9b055ea9f21d74cb168ef0277f39de00d15dab2a9af0e7b4c98da0c5c0516c14 Deleted: sha256:7a3378ae3958bfa7bd8bac75f11995dfbb3c277a735e650a621714603a47a7fa Untagged: cyb3rward0g/helk-zookeeper:2.2.0 Untagged: cyb3rward0g/helk-zookeeper@sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Deleted: sha256:fa0e4ceb4a83035bd5d80d84cd8827f468821a4178b19df675c135a382c98357 Deleted: sha256:b98cfe820a642bfb0eaaf25bd78f374dc387f78bc19dbdcef67f687d05294c99 Deleted: sha256:abaec1e946404eda0c521c8eb4183450cb341d1098ec3799a5718961dbcd52c5 Deleted: sha256:3a77b7249027c7f216acaaecefcd8241836f172c695cbdf4230491d870153478 Deleted: sha256:a7a618294575059f3ec2fb28a2773cc519e2060479f8af32a78cdbc8e5e2283c Deleted: sha256:f6ea8491f794f0200308a11942febda354d3ccb7dbd8b959e56b8a57c45d8953 Untagged: cyb3rward0g/helk-nginx:0.0.7 Untagged: cyb3rward0g/helk-nginx@sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Deleted: sha256:280d044b6719787259c29053d92815e5e11fed4b946d227f84d71ac3c0ec228c Deleted: sha256:90e53da57e1792c0f049cfea1d38f4ac3df3529eefab24ac7e0bea54acbdb426 Deleted: sha256:717f3f4f791b01b6d2926fbb62fe3d5925b31350326ddb68c9f5f636a9c0f64d Deleted: sha256:5d772798c7e2b92a62cf7f07687b67748e277f50e3f313ed0490bfd176408e6b Deleted: sha256:02aaf8287515441c0bfd148e86821a2dc3f711a296044265e42011022ff2664c Deleted: sha256:9a5c1086828b3777b6c1fff86e3437f563be17695ad421397c0a7a0e5a5783b3 Deleted: sha256:762bd884fbde30e58308cd8a21aa2e59dfef049e756015366ca808e5da1a6a59 Deleted: sha256:8961c31eff48193478910b31c7710b7ee57c41e4f12491b6088140cfc119be26 Deleted: sha256:e8b40fc220ff63ad06c8fc48a97bcc06a6e6e344141ca3f5eb0023a01ccfc070 Deleted: sha256:4f0db2f651828b41116784d7299fa130c817808867c092c74de1bd63861fd564 Deleted: sha256:d03c684c7dd711982dddd1ab0e622440c45d27b5d015020e0aef65be0c2e7c9b Deleted: sha256:63eed5473967cb9a5a4c98a5b1d2b6013d368bf88836aab99b5c5178c8f2f51f Deleted: sha256:86267d11f0c14fca869691b9b32bdd610b6ab8d9033d59ee64bdcc2cf0219bce Deleted: sha256:d9a8b3f912eee0b322b86fa0f6888558a468c384611c71178987b20e3a0ebafc Deleted: sha256:4e627d1476f22151f05e5214147d6cc6e03ad79a082f01aca6560aa75c7ade3a Deleted: sha256:757b76a12baba45fcbe76abbdd99723be9d94c12a2ad40354dc49ff5fbe1f5c1 Deleted: sha256:f49017d4d5ce9c0f544c82ed5cbc0672fbcb593be77f954891b22b4d0d4c0a84 Error response from daemon: conflict: unable to delete 785f91d9f484 (cannot be forced) - image is being used by running container c3ac42108fd1 Error response from daemon: conflict: unable to delete 5901cec02503 (cannot be forced) - image is being used by running container 61a8d043c4f6 Error response from daemon: conflict: unable to delete 93ae8cd11560 (cannot be forced) - image is being used by running container f7313ed22d31 Error response from daemon: conflict: unable to delete 714b175e84e8 (cannot be forced) - image is being used by running container 22e7070b53f9 Error response from daemon: conflict: unable to delete 12ad640a1ec0 (cannot be forced) - image is being used by running container 7ee92ef1d188 "docker stop" requires at least 1 argument. See 'docker stop --help'. Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] Stop one or more running containers "docker stop" requires at least 1 argument. See 'docker stop --help'. Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] Stop one or more running containers "docker stop" requires at least 1 argument. See 'docker stop --help'. Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] Stop one or more running containers Adding password for user helk Pulling helk-nginx (cyb3rward0g/helk-nginx:0.0.7)... 0.0.7: Pulling from cyb3rward0g/helk-nginx Digest: sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Status: Downloaded newer image for cyb3rward0g/helk-nginx:0.0.7 Pulling helk-zookeeper (cyb3rward0g/helk-zookeeper:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-zookeeper Digest: sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Status: Downloaded newer image for cyb3rward0g/helk-zookeeper:2.2.0 Pulling helk-kafka-broker (cyb3rward0g/helk-kafka-broker:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-kafka-broker Digest: sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Status: Downloaded newer image for cyb3rward0g/helk-kafka-broker:2.2.0 Pulling helk-elastalert (cyb3rward0g/helk-elastalert:0.2.4)... 0.2.4: Pulling from cyb3rward0g/helk-elastalert Digest: sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Status: Downloaded newer image for cyb3rward0g/helk-elastalert:0.2.4 Starting helk-elasticsearch ... [1A[2K Starting helk-elasticsearch ... [32mdone[0m [1BStarting helk-kibana ... [1A[2K Starting helk-kibana ... [32mdone[0m [1BStarting helk-logstash ... Creating helk-nginx ... [2A[2K Starting helk-logstash ... [32mdone[0m [2BCreating helk-zookeeper ... Creating helk-elastalert ... [3A[2K Creating helk-nginx ... [32mdone[0m [3B[2A[2K Creating helk-zookeeper ... [32mdone[0m [2BCreating helk-kafka-broker ... [2A[2K Creating helk-elastalert ... [32mdone[0m [2B[1A[2K Creating helk-kafka-broker ... [32mdone[0m [1BRecreating helk-ksql-server ... [1A[2K Recreating helk-ksql-server ... [32mdone[0m [1BRecreating helk-ksql-cli ... [1A[2K Recreating helk-ksql-cli ... [32mdone[0m [1B33deda66e73a a91e80d0da9d f7bb556d86b4 9632209fe79b 33deda66e73a a91e80d0da9d f7bb556d86b4 9632209fe79b Untagged: cyb3rward0g/helk-elastalert:0.2.4 Untagged: cyb3rward0g/helk-elastalert@sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Deleted: sha256:c7d5d7eed99cce73b7073770dc3c6ecc56a0a447305027ece73e4fabfa0ca51c Deleted: sha256:7ddfedd66acd6b1603c988457aa5cd33417affc4fd9d2e3d768e7ef327bd87e5 Deleted: sha256:19f62ea43ccbd84698f2a9375d08a20d52a02e7f4788843a4d5390cabd3c2472 Deleted: sha256:255396df8ad9053ed73d5a95c12d855e52ad967cd9054676efcfcd85310a1882 Deleted: sha256:df064f144aab0c435fcb883babd6597b724d8f2b73373976f4d3ed964fbbdcee Deleted: sha256:91fe17dde3260a0caf19608ce8e57a69ca0b16ac42e16d489cd08ddb48cb53e9 Deleted: sha256:6872aedee364733b71cd03b5e315ac483f9e6a0bfd61e9a15ca8c3fe7e539936 Untagged: cyb3rward0g/helk-kafka-broker:2.2.0 Untagged: cyb3rward0g/helk-kafka-broker@sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Deleted: sha256:c2f7cd82ae1da69dd48c2fc2c26ae855fc8c05efb9e7a8f5d985ebd6c99fddd7 Deleted: sha256:b9059b735d4f7ab0ab47f76fceee5c10d6674739f7d3b2cb89c6cf5db0f92293 Deleted: sha256:52b06347f9930845994d3a549cc9ff99524b3a6ce63ed35b85fa67aa7955ecf8 Deleted: sha256:9b055ea9f21d74cb168ef0277f39de00d15dab2a9af0e7b4c98da0c5c0516c14 Deleted: sha256:7a3378ae3958bfa7bd8bac75f11995dfbb3c277a735e650a621714603a47a7fa Untagged: cyb3rward0g/helk-zookeeper:2.2.0 Untagged: cyb3rward0g/helk-zookeeper@sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Deleted: sha256:fa0e4ceb4a83035bd5d80d84cd8827f468821a4178b19df675c135a382c98357 Deleted: sha256:b98cfe820a642bfb0eaaf25bd78f374dc387f78bc19dbdcef67f687d05294c99 Deleted: sha256:abaec1e946404eda0c521c8eb4183450cb341d1098ec3799a5718961dbcd52c5 Deleted: sha256:3a77b7249027c7f216acaaecefcd8241836f172c695cbdf4230491d870153478 Deleted: sha256:a7a618294575059f3ec2fb28a2773cc519e2060479f8af32a78cdbc8e5e2283c Deleted: sha256:f6ea8491f794f0200308a11942febda354d3ccb7dbd8b959e56b8a57c45d8953 Untagged: cyb3rward0g/helk-nginx:0.0.7 Untagged: cyb3rward0g/helk-nginx@sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Deleted: sha256:280d044b6719787259c29053d92815e5e11fed4b946d227f84d71ac3c0ec228c Deleted: sha256:90e53da57e1792c0f049cfea1d38f4ac3df3529eefab24ac7e0bea54acbdb426 Deleted: sha256:717f3f4f791b01b6d2926fbb62fe3d5925b31350326ddb68c9f5f636a9c0f64d Deleted: sha256:5d772798c7e2b92a62cf7f07687b67748e277f50e3f313ed0490bfd176408e6b Deleted: sha256:02aaf8287515441c0bfd148e86821a2dc3f711a296044265e42011022ff2664c Deleted: sha256:9a5c1086828b3777b6c1fff86e3437f563be17695ad421397c0a7a0e5a5783b3 Deleted: sha256:762bd884fbde30e58308cd8a21aa2e59dfef049e756015366ca808e5da1a6a59 Deleted: sha256:8961c31eff48193478910b31c7710b7ee57c41e4f12491b6088140cfc119be26 Deleted: sha256:e8b40fc220ff63ad06c8fc48a97bcc06a6e6e344141ca3f5eb0023a01ccfc070 Deleted: sha256:4f0db2f651828b41116784d7299fa130c817808867c092c74de1bd63861fd564 Deleted: sha256:d03c684c7dd711982dddd1ab0e622440c45d27b5d015020e0aef65be0c2e7c9b Deleted: sha256:63eed5473967cb9a5a4c98a5b1d2b6013d368bf88836aab99b5c5178c8f2f51f Deleted: sha256:86267d11f0c14fca869691b9b32bdd610b6ab8d9033d59ee64bdcc2cf0219bce Deleted: sha256:d9a8b3f912eee0b322b86fa0f6888558a468c384611c71178987b20e3a0ebafc Deleted: sha256:4e627d1476f22151f05e5214147d6cc6e03ad79a082f01aca6560aa75c7ade3a Deleted: sha256:757b76a12baba45fcbe76abbdd99723be9d94c12a2ad40354dc49ff5fbe1f5c1 Deleted: sha256:f49017d4d5ce9c0f544c82ed5cbc0672fbcb593be77f954891b22b4d0d4c0a84 Error response from daemon: conflict: unable to delete 785f91d9f484 (cannot be forced) - image is being used by running container b148a7e28118 Error response from daemon: conflict: unable to delete 5901cec02503 (cannot be forced) - image is being used by running container 03a56527b683 Error response from daemon: conflict: unable to delete 93ae8cd11560 (cannot be forced) - image is being used by running container f7313ed22d31 Error response from daemon: conflict: unable to delete 714b175e84e8 (cannot be forced) - image is being used by running container 22e7070b53f9 Error response from daemon: conflict: unable to delete 12ad640a1ec0 (cannot be forced) - image is being used by running container 7ee92ef1d188 Adding password for user helk Pulling helk-elasticsearch (docker.elastic.co/elasticsearch/elasticsearch:7.1.0)... 7.1.0: Pulling from elasticsearch/elasticsearch Digest: sha256:802b6a299260dbaf21a9c57e3a634491ff788a1ea13a51598d4cd105739509c4 Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.1.0 Pulling helk-kibana (docker.elastic.co/kibana/kibana:7.1.0)... 7.1.0: Pulling from kibana/kibana Digest: sha256:0eb53e2eb9a8846a5a4e700115a4b6e207a8f298128a45cdd28a514f708ec0f3 Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.1.0 Pulling helk-logstash (docker.elastic.co/logstash/logstash:7.1.0)... 7.1.0: Pulling from logstash/logstash Digest: sha256:9258bd2ef10c084a267e470e4fac7b5144e2bd932d628deab4f2c8cc2ff47dd0 Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.1.0 Pulling helk-nginx (cyb3rward0g/helk-nginx:0.0.7)... 0.0.7: Pulling from cyb3rward0g/helk-nginx Digest: sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Status: Downloaded newer image for cyb3rward0g/helk-nginx:0.0.7 Pulling helk-zookeeper (cyb3rward0g/helk-zookeeper:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-zookeeper Digest: sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Status: Downloaded newer image for cyb3rward0g/helk-zookeeper:2.2.0 Pulling helk-kafka-broker (cyb3rward0g/helk-kafka-broker:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-kafka-broker Digest: sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Status: Downloaded newer image for cyb3rward0g/helk-kafka-broker:2.2.0 Pulling helk-ksql-server (confluentinc/cp-ksql-server:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-server Digest: sha256:063add111cc93b1a0118f88b577e31303045d4cc08eb1d21458429f05cba4b02 Status: Downloaded newer image for confluentinc/cp-ksql-server:5.1.3 Pulling helk-ksql-cli (confluentinc/cp-ksql-cli:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-cli Digest: sha256:18c0ccb00fbf87679e16e9e0da600548fcb236a2fd173263b09e89b2d3a42cc3 Status: Downloaded newer image for confluentinc/cp-ksql-cli:5.1.3 Pulling helk-elastalert (cyb3rward0g/helk-elastalert:0.2.4)... 0.2.4: Pulling from cyb3rward0g/helk-elastalert Digest: sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Status: Downloaded newer image for cyb3rward0g/helk-elastalert:0.2.4 Creating helk-elasticsearch ... [1A[2K Creating helk-elasticsearch ... [32mdone[0m [1BCreating helk-kibana ... [1A[2K Creating helk-kibana ... [32mdone[0m [1BCreating helk-nginx ... Creating helk-logstash ... [2A[2K Creating helk-nginx ... [32mdone[0m [2B[1A[2K Creating helk-logstash ... [32mdone[0m [1BCreating helk-elastalert ... Creating helk-zookeeper ... [2A[2K Creating helk-elastalert ... [32mdone[0m [2B[1A[2K Creating helk-zookeeper ... [32mdone[0m [1BCreating helk-kafka-broker ... [1A[2K Creating helk-kafka-broker ... [32mdone[0m [1BCreating helk-ksql-server ... [1A[2K Creating helk-ksql-server ... [32mdone[0m [1BCreating helk-ksql-cli ... [1A[2K Creating helk-ksql-cli ... [32mdone[0m [1Ba268fb4c59d6 d2a8409ecc99 a02a2e4226e7 d87ccecfa214 a268fb4c59d6 d2a8409ecc99 a02a2e4226e7 d87ccecfa214 Untagged: cyb3rward0g/helk-elastalert:0.2.4 Untagged: cyb3rward0g/helk-elastalert@sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Deleted: sha256:c7d5d7eed99cce73b7073770dc3c6ecc56a0a447305027ece73e4fabfa0ca51c Deleted: sha256:7ddfedd66acd6b1603c988457aa5cd33417affc4fd9d2e3d768e7ef327bd87e5 Deleted: sha256:19f62ea43ccbd84698f2a9375d08a20d52a02e7f4788843a4d5390cabd3c2472 Deleted: sha256:255396df8ad9053ed73d5a95c12d855e52ad967cd9054676efcfcd85310a1882 Deleted: sha256:df064f144aab0c435fcb883babd6597b724d8f2b73373976f4d3ed964fbbdcee Deleted: sha256:91fe17dde3260a0caf19608ce8e57a69ca0b16ac42e16d489cd08ddb48cb53e9 Deleted: sha256:6872aedee364733b71cd03b5e315ac483f9e6a0bfd61e9a15ca8c3fe7e539936 Untagged: cyb3rward0g/helk-kafka-broker:2.2.0 Untagged: cyb3rward0g/helk-kafka-broker@sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Deleted: sha256:c2f7cd82ae1da69dd48c2fc2c26ae855fc8c05efb9e7a8f5d985ebd6c99fddd7 Deleted: sha256:b9059b735d4f7ab0ab47f76fceee5c10d6674739f7d3b2cb89c6cf5db0f92293 Deleted: sha256:52b06347f9930845994d3a549cc9ff99524b3a6ce63ed35b85fa67aa7955ecf8 Deleted: sha256:9b055ea9f21d74cb168ef0277f39de00d15dab2a9af0e7b4c98da0c5c0516c14 Deleted: sha256:7a3378ae3958bfa7bd8bac75f11995dfbb3c277a735e650a621714603a47a7fa Untagged: cyb3rward0g/helk-zookeeper:2.2.0 Untagged: cyb3rward0g/helk-zookeeper@sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Deleted: sha256:fa0e4ceb4a83035bd5d80d84cd8827f468821a4178b19df675c135a382c98357 Deleted: sha256:b98cfe820a642bfb0eaaf25bd78f374dc387f78bc19dbdcef67f687d05294c99 Deleted: sha256:abaec1e946404eda0c521c8eb4183450cb341d1098ec3799a5718961dbcd52c5 Deleted: sha256:3a77b7249027c7f216acaaecefcd8241836f172c695cbdf4230491d870153478 Deleted: sha256:a7a618294575059f3ec2fb28a2773cc519e2060479f8af32a78cdbc8e5e2283c Deleted: sha256:f6ea8491f794f0200308a11942febda354d3ccb7dbd8b959e56b8a57c45d8953 Untagged: cyb3rward0g/helk-nginx:0.0.7 Untagged: cyb3rward0g/helk-nginx@sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Deleted: sha256:280d044b6719787259c29053d92815e5e11fed4b946d227f84d71ac3c0ec228c Deleted: sha256:90e53da57e1792c0f049cfea1d38f4ac3df3529eefab24ac7e0bea54acbdb426 Deleted: sha256:717f3f4f791b01b6d2926fbb62fe3d5925b31350326ddb68c9f5f636a9c0f64d Deleted: sha256:5d772798c7e2b92a62cf7f07687b67748e277f50e3f313ed0490bfd176408e6b Deleted: sha256:02aaf8287515441c0bfd148e86821a2dc3f711a296044265e42011022ff2664c Deleted: sha256:9a5c1086828b3777b6c1fff86e3437f563be17695ad421397c0a7a0e5a5783b3 Deleted: sha256:762bd884fbde30e58308cd8a21aa2e59dfef049e756015366ca808e5da1a6a59 Deleted: sha256:8961c31eff48193478910b31c7710b7ee57c41e4f12491b6088140cfc119be26 Deleted: sha256:e8b40fc220ff63ad06c8fc48a97bcc06a6e6e344141ca3f5eb0023a01ccfc070 Deleted: sha256:4f0db2f651828b41116784d7299fa130c817808867c092c74de1bd63861fd564 Deleted: sha256:d03c684c7dd711982dddd1ab0e622440c45d27b5d015020e0aef65be0c2e7c9b Deleted: sha256:63eed5473967cb9a5a4c98a5b1d2b6013d368bf88836aab99b5c5178c8f2f51f Deleted: sha256:86267d11f0c14fca869691b9b32bdd610b6ab8d9033d59ee64bdcc2cf0219bce Deleted: sha256:d9a8b3f912eee0b322b86fa0f6888558a468c384611c71178987b20e3a0ebafc Deleted: sha256:4e627d1476f22151f05e5214147d6cc6e03ad79a082f01aca6560aa75c7ade3a Deleted: sha256:757b76a12baba45fcbe76abbdd99723be9d94c12a2ad40354dc49ff5fbe1f5c1 Deleted: sha256:f49017d4d5ce9c0f544c82ed5cbc0672fbcb593be77f954891b22b4d0d4c0a84 Error response from daemon: conflict: unable to delete 785f91d9f484 (cannot be forced) - image is being used by running container 3b8f4b16ee07 Error response from daemon: conflict: unable to delete 5901cec02503 (cannot be forced) - image is being used by running container 26ff8fb43241 Error response from daemon: conflict: unable to delete 93ae8cd11560 (cannot be forced) - image is being used by running container 6e8dd919f3ed Error response from daemon: conflict: unable to delete 714b175e84e8 (cannot be forced) - image is being used by running container 7290a5cd257d Error response from daemon: conflict: unable to delete 12ad640a1ec0 (cannot be forced) - image is being used by running container 3083419626d2 "docker stop" requires at least 1 argument. See 'docker stop --help'. Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] Stop one or more running containers "docker stop" requires at least 1 argument. See 'docker stop --help'. Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] Stop one or more running containers

neu5ron commented 5 years ago

thanks for sending. are you using docker images for anything else on this machine?

On Sat, Sep 7, 2019 at 9:33 AM sugarp0pe notifications@github.com wrote:

Thank you for quick response helk-install.log

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists... Building dependency tree... Reading state information... The following additional packages will be installed: libapr1 libaprutil1 The following NEW packages will be installed: apache2-utils libapr1 libaprutil1 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. Need to get 259 kB of archives. After this operation, 865 kB of additional disk space will be used. Get:1 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libapr1 amd64 1.6.3-2 [90.9 kB] Get:2 http://us.archive.ubuntu.com/ubuntu bionic/main amd64 libaprutil1 amd64 1.6.1-2 [84.4 kB] Get:3 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 apache2-utils amd64 2.4.29-1ubuntu4.10 [83.9 kB] Fetched 259 kB in 3s (103 kB/s) Selecting previously unselected package libapr1:amd64. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102895 files and directories currently installed.) Preparing to unpack .../libapr1_1.6.3-2_amd64.deb ... Unpacking libapr1:amd64 (1.6.3-2) ... Selecting previously unselected package libaprutil1:amd64. Preparing to unpack .../libaprutil1_1.6.1-2_amd64.deb ... Unpacking libaprutil1:amd64 (1.6.1-2) ... Selecting previously unselected package apache2-utils. Preparing to unpack .../apache2-utils_2.4.29-1ubuntu4.10_amd64.deb ... Unpacking apache2-utils (2.4.29-1ubuntu4.10) ... Setting up libapr1:amd64 (1.6.3-2) ... Setting up libaprutil1:amd64 (1.6.1-2) ... Setting up apache2-utils (2.4.29-1ubuntu4.10) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Processing triggers for libc-bin (2.27-3ubuntu1) ... Adding password for user helk Creating network "docker_helk" with driver "bridge" Creating volume "docker_esdata" with local driver

Pulling helk-elasticsearch ( docker.elastic.co/elasticsearch/elasticsearch:7.1.0)... 7.1.0: Pulling from elasticsearch/elasticsearch Digest: sha256:802b6a299260dbaf21a9c57e3a634491ff788a1ea13a51598d4cd105739509c4 Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.1.0 Pulling helk-kibana (docker.elastic.co/kibana/kibana:7.1.0)... 7.1.0: Pulling from kibana/kibana Digest: sha256:0eb53e2eb9a8846a5a4e700115a4b6e207a8f298128a45cdd28a514f708ec0f3 Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.1.0 Pulling helk-logstash (docker.elastic.co/logstash/logstash:7.1.0)... 7.1.0: Pulling from logstash/logstash Digest: sha256:9258bd2ef10c084a267e470e4fac7b5144e2bd932d628deab4f2c8cc2ff47dd0 Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.1.0 Pulling helk-nginx (cyb3rward0g/helk-nginx:0.0.7)... 0.0.7: Pulling from cyb3rward0g/helk-nginx Digest: sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Status: Downloaded newer image for cyb3rward0g/helk-nginx:0.0.7 Pulling helk-zookeeper (cyb3rward0g/helk-zookeeper:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-zookeeper Digest: sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Status: Downloaded newer image for cyb3rward0g/helk-zookeeper:2.2.0 Pulling helk-kafka-broker (cyb3rward0g/helk-kafka-broker:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-kafka-broker Digest: sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Status: Downloaded newer image for cyb3rward0g/helk-kafka-broker:2.2.0 Pulling helk-ksql-server (confluentinc/cp-ksql-server:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-server Digest: sha256:063add111cc93b1a0118f88b577e31303045d4cc08eb1d21458429f05cba4b02 Status: Downloaded newer image for confluentinc/cp-ksql-server:5.1.3 Pulling helk-ksql-cli (confluentinc/cp-ksql-cli:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-cli Digest: sha256:18c0ccb00fbf87679e16e9e0da600548fcb236a2fd173263b09e89b2d3a42cc3 Status: Downloaded newer image for confluentinc/cp-ksql-cli:5.1.3

Pulling helk-elastalert (cyb3rward0g/helk-elastalert:0.2.4)... 0.2.4: Pulling from cyb3rward0g/helk-elastalert Digest: sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Status: Downloaded newer image for cyb3rward0g/helk-elastalert:0.2.4 Creating helk-elasticsearch ... �[1A�[2K Creating helk-elasticsearch ... �[32mdone�[0m �[1BCreating helk-kibana ... �[1A�[2K Creating helk-kibana ... �[32mdone�[0m �[1BCreating helk-logstash ... Creating helk-nginx ... �[2A�[2K Creating helk-logstash ... �[32mdone�[0m �[2BCreating helk-elastalert ... Creating helk-zookeeper ... �[3A�[2K Creating helk-nginx ... �[32mdone�[0m �[3B�[2A�[2K Creating helk-elastalert ... �[32mdone�[0m �[2B�[1A�[2K Creating helk-zookeeper ... �[32mdone�[0m �[1BCreating helk-kafka-broker ... �[1A�[2K Creating helk-kafka-broker ... �[32mdone�[0m �[1BCreating helk-ksql-server ... �[1A�[2K Creating helk-ksql-server ... �[32mdone�[0m �[1BCreating helk-ksql-cli ... �[1A�[2K Creating helk-ksql-cli ... �[32mdone�[0m �[1BAdding password for user helk helk-elasticsearch is up-to-date helk-kibana is up-to-date helk-logstash is up-to-date helk-nginx is up-to-date helk-elastalert is up-to-date helk-zookeeper is up-to-date helk-kafka-broker is up-to-date helk-ksql-server is up-to-date helk-ksql-cli is up-to-date 885d61b0f8a5 5b868662e349 a29fd6d11d79 a45a104eca2b 885d61b0f8a5 5b868662e349 a29fd6d11d79 a45a104eca2b Untagged: cyb3rward0g/helk-elastalert:0.2.4 Untagged: cyb3rward0g/helk-elastalert@sha256 :dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Deleted: sha256:c7d5d7eed99cce73b7073770dc3c6ecc56a0a447305027ece73e4fabfa0ca51c Deleted: sha256:7ddfedd66acd6b1603c988457aa5cd33417affc4fd9d2e3d768e7ef327bd87e5 Deleted: sha256:19f62ea43ccbd84698f2a9375d08a20d52a02e7f4788843a4d5390cabd3c2472 Deleted: sha256:255396df8ad9053ed73d5a95c12d855e52ad967cd9054676efcfcd85310a1882 Deleted: sha256:df064f144aab0c435fcb883babd6597b724d8f2b73373976f4d3ed964fbbdcee Deleted: sha256:91fe17dde3260a0caf19608ce8e57a69ca0b16ac42e16d489cd08ddb48cb53e9 Deleted: sha256:6872aedee364733b71cd03b5e315ac483f9e6a0bfd61e9a15ca8c3fe7e539936 Untagged: cyb3rward0g/helk-kafka-broker:2.2.0 Untagged: cyb3rward0g/helk-kafka-broker@sha256 :8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Deleted: sha256:c2f7cd82ae1da69dd48c2fc2c26ae855fc8c05efb9e7a8f5d985ebd6c99fddd7 Deleted: sha256:b9059b735d4f7ab0ab47f76fceee5c10d6674739f7d3b2cb89c6cf5db0f92293 Deleted: sha256:52b06347f9930845994d3a549cc9ff99524b3a6ce63ed35b85fa67aa7955ecf8 Deleted: sha256:9b055ea9f21d74cb168ef0277f39de00d15dab2a9af0e7b4c98da0c5c0516c14 Deleted: sha256:7a3378ae3958bfa7bd8bac75f11995dfbb3c277a735e650a621714603a47a7fa Untagged: cyb3rward0g/helk-zookeeper:2.2.0 Untagged: cyb3rward0g/helk-zookeeper@sha256 :a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Deleted: sha256:fa0e4ceb4a83035bd5d80d84cd8827f468821a4178b19df675c135a382c98357 Deleted: sha256:b98cfe820a642bfb0eaaf25bd78f374dc387f78bc19dbdcef67f687d05294c99 Deleted: sha256:abaec1e946404eda0c521c8eb4183450cb341d1098ec3799a5718961dbcd52c5 Deleted: sha256:3a77b7249027c7f216acaaecefcd8241836f172c695cbdf4230491d870153478 Deleted: sha256:a7a618294575059f3ec2fb28a2773cc519e2060479f8af32a78cdbc8e5e2283c Deleted: sha256:f6ea8491f794f0200308a11942febda354d3ccb7dbd8b959e56b8a57c45d8953 Untagged: cyb3rward0g/helk-nginx:0.0.7 Untagged: cyb3rward0g/helk-nginx@sha256 :8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Deleted: sha256:280d044b6719787259c29053d92815e5e11fed4b946d227f84d71ac3c0ec228c Deleted: sha256:90e53da57e1792c0f049cfea1d38f4ac3df3529eefab24ac7e0bea54acbdb426 Deleted: sha256:717f3f4f791b01b6d2926fbb62fe3d5925b31350326ddb68c9f5f636a9c0f64d Deleted: sha256:5d772798c7e2b92a62cf7f07687b67748e277f50e3f313ed0490bfd176408e6b Deleted: sha256:02aaf8287515441c0bfd148e86821a2dc3f711a296044265e42011022ff2664c Deleted: sha256:9a5c1086828b3777b6c1fff86e3437f563be17695ad421397c0a7a0e5a5783b3 Deleted: sha256:762bd884fbde30e58308cd8a21aa2e59dfef049e756015366ca808e5da1a6a59 Deleted: sha256:8961c31eff48193478910b31c7710b7ee57c41e4f12491b6088140cfc119be26 Deleted: sha256:e8b40fc220ff63ad06c8fc48a97bcc06a6e6e344141ca3f5eb0023a01ccfc070 Deleted: sha256:4f0db2f651828b41116784d7299fa130c817808867c092c74de1bd63861fd564 Deleted: sha256:d03c684c7dd711982dddd1ab0e622440c45d27b5d015020e0aef65be0c2e7c9b Deleted: sha256:63eed5473967cb9a5a4c98a5b1d2b6013d368bf88836aab99b5c5178c8f2f51f Deleted: sha256:86267d11f0c14fca869691b9b32bdd610b6ab8d9033d59ee64bdcc2cf0219bce Deleted: sha256:d9a8b3f912eee0b322b86fa0f6888558a468c384611c71178987b20e3a0ebafc Deleted: sha256:4e627d1476f22151f05e5214147d6cc6e03ad79a082f01aca6560aa75c7ade3a Deleted: sha256:757b76a12baba45fcbe76abbdd99723be9d94c12a2ad40354dc49ff5fbe1f5c1 Deleted: sha256:f49017d4d5ce9c0f544c82ed5cbc0672fbcb593be77f954891b22b4d0d4c0a84 Error response from daemon: conflict: unable to delete 785f91d9f484 (cannot be forced) - image is being used by running container c3ac42108fd1 Error response from daemon: conflict: unable to delete 5901cec02503 (cannot be forced) - image is being used by running container 61a8d043c4f6 Error response from daemon: conflict: unable to delete 93ae8cd11560 (cannot be forced) - image is being used by running container f7313ed22d31 Error response from daemon: conflict: unable to delete 714b175e84e8 (cannot be forced) - image is being used by running container 22e7070b53f9 Error response from daemon: conflict: unable to delete 12ad640a1ec0 (cannot be forced) - image is being used by running container 7ee92ef1d188 "docker stop" requires at least 1 argument. See 'docker stop --help'.

Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...]

Stop one or more running containers "docker stop" requires at least 1 argument. See 'docker stop --help'.

Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...]

Stop one or more running containers "docker stop" requires at least 1 argument. See 'docker stop --help'.

Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...]

Stop one or more running containers Adding password for user helk Pulling helk-nginx (cyb3rward0g/helk-nginx:0.0.7)... 0.0.7: Pulling from cyb3rward0g/helk-nginx Digest: sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Status: Downloaded newer image for cyb3rward0g/helk-nginx:0.0.7 Pulling helk-zookeeper (cyb3rward0g/helk-zookeeper:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-zookeeper Digest: sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Status: Downloaded newer image for cyb3rward0g/helk-zookeeper:2.2.0 Pulling helk-kafka-broker (cyb3rward0g/helk-kafka-broker:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-kafka-broker Digest: sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Status: Downloaded newer image for cyb3rward0g/helk-kafka-broker:2.2.0 Pulling helk-elastalert (cyb3rward0g/helk-elastalert:0.2.4)... 0.2.4: Pulling from cyb3rward0g/helk-elastalert Digest: sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Status: Downloaded newer image for cyb3rward0g/helk-elastalert:0.2.4 Starting helk-elasticsearch ... �[1A�[2K Starting helk-elasticsearch ... �[32mdone�[0m �[1BStarting helk-kibana ... �[1A�[2K Starting helk-kibana ... �[32mdone�[0m �[1BStarting helk-logstash ... Creating helk-nginx ... �[2A�[2K Starting helk-logstash ... �[32mdone�[0m �[2BCreating helk-zookeeper ... Creating helk-elastalert ... �[3A�[2K Creating helk-nginx ... �[32mdone�[0m �[3B�[2A�[2K Creating helk-zookeeper ... �[32mdone�[0m �[2BCreating helk-kafka-broker ... �[2A�[2K Creating helk-elastalert ... �[32mdone�[0m �[2B�[1A�[2K Creating helk-kafka-broker ... �[32mdone�[0m �[1BRecreating helk-ksql-server ... �[1A�[2K Recreating helk-ksql-server ... �[32mdone�[0m �[1BRecreating helk-ksql-cli ... �[1A�[2K Recreating helk-ksql-cli ... �[32mdone�[0m �[1B33deda66e73a a91e80d0da9d f7bb556d86b4 9632209fe79b 33deda66e73a a91e80d0da9d f7bb556d86b4 9632209fe79b Untagged: cyb3rward0g/helk-elastalert:0.2.4 Untagged: cyb3rward0g/helk-elastalert@sha256 :dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Deleted: sha256:c7d5d7eed99cce73b7073770dc3c6ecc56a0a447305027ece73e4fabfa0ca51c Deleted: sha256:7ddfedd66acd6b1603c988457aa5cd33417affc4fd9d2e3d768e7ef327bd87e5 Deleted: sha256:19f62ea43ccbd84698f2a9375d08a20d52a02e7f4788843a4d5390cabd3c2472 Deleted: sha256:255396df8ad9053ed73d5a95c12d855e52ad967cd9054676efcfcd85310a1882 Deleted: sha256:df064f144aab0c435fcb883babd6597b724d8f2b73373976f4d3ed964fbbdcee Deleted: sha256:91fe17dde3260a0caf19608ce8e57a69ca0b16ac42e16d489cd08ddb48cb53e9 Deleted: sha256:6872aedee364733b71cd03b5e315ac483f9e6a0bfd61e9a15ca8c3fe7e539936 Untagged: cyb3rward0g/helk-kafka-broker:2.2.0 Untagged: cyb3rward0g/helk-kafka-broker@sha256 :8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Deleted: sha256:c2f7cd82ae1da69dd48c2fc2c26ae855fc8c05efb9e7a8f5d985ebd6c99fddd7 Deleted: sha256:b9059b735d4f7ab0ab47f76fceee5c10d6674739f7d3b2cb89c6cf5db0f92293 Deleted: sha256:52b06347f9930845994d3a549cc9ff99524b3a6ce63ed35b85fa67aa7955ecf8 Deleted: sha256:9b055ea9f21d74cb168ef0277f39de00d15dab2a9af0e7b4c98da0c5c0516c14 Deleted: sha256:7a3378ae3958bfa7bd8bac75f11995dfbb3c277a735e650a621714603a47a7fa Untagged: cyb3rward0g/helk-zookeeper:2.2.0 Untagged: cyb3rward0g/helk-zookeeper@sha256 :a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Deleted: sha256:fa0e4ceb4a83035bd5d80d84cd8827f468821a4178b19df675c135a382c98357 Deleted: sha256:b98cfe820a642bfb0eaaf25bd78f374dc387f78bc19dbdcef67f687d05294c99 Deleted: sha256:abaec1e946404eda0c521c8eb4183450cb341d1098ec3799a5718961dbcd52c5 Deleted: sha256:3a77b7249027c7f216acaaecefcd8241836f172c695cbdf4230491d870153478 Deleted: sha256:a7a618294575059f3ec2fb28a2773cc519e2060479f8af32a78cdbc8e5e2283c Deleted: sha256:f6ea8491f794f0200308a11942febda354d3ccb7dbd8b959e56b8a57c45d8953 Untagged: cyb3rward0g/helk-nginx:0.0.7 Untagged: cyb3rward0g/helk-nginx@sha256 :8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Deleted: sha256:280d044b6719787259c29053d92815e5e11fed4b946d227f84d71ac3c0ec228c Deleted: sha256:90e53da57e1792c0f049cfea1d38f4ac3df3529eefab24ac7e0bea54acbdb426 Deleted: sha256:717f3f4f791b01b6d2926fbb62fe3d5925b31350326ddb68c9f5f636a9c0f64d Deleted: sha256:5d772798c7e2b92a62cf7f07687b67748e277f50e3f313ed0490bfd176408e6b Deleted: sha256:02aaf8287515441c0bfd148e86821a2dc3f711a296044265e42011022ff2664c Deleted: sha256:9a5c1086828b3777b6c1fff86e3437f563be17695ad421397c0a7a0e5a5783b3 Deleted: sha256:762bd884fbde30e58308cd8a21aa2e59dfef049e756015366ca808e5da1a6a59 Deleted: sha256:8961c31eff48193478910b31c7710b7ee57c41e4f12491b6088140cfc119be26 Deleted: sha256:e8b40fc220ff63ad06c8fc48a97bcc06a6e6e344141ca3f5eb0023a01ccfc070 Deleted: sha256:4f0db2f651828b41116784d7299fa130c817808867c092c74de1bd63861fd564 Deleted: sha256:d03c684c7dd711982dddd1ab0e622440c45d27b5d015020e0aef65be0c2e7c9b Deleted: sha256:63eed5473967cb9a5a4c98a5b1d2b6013d368bf88836aab99b5c5178c8f2f51f Deleted: sha256:86267d11f0c14fca869691b9b32bdd610b6ab8d9033d59ee64bdcc2cf0219bce Deleted: sha256:d9a8b3f912eee0b322b86fa0f6888558a468c384611c71178987b20e3a0ebafc Deleted: sha256:4e627d1476f22151f05e5214147d6cc6e03ad79a082f01aca6560aa75c7ade3a Deleted: sha256:757b76a12baba45fcbe76abbdd99723be9d94c12a2ad40354dc49ff5fbe1f5c1 Deleted: sha256:f49017d4d5ce9c0f544c82ed5cbc0672fbcb593be77f954891b22b4d0d4c0a84 Error response from daemon: conflict: unable to delete 785f91d9f484 (cannot be forced) - image is being used by running container b148a7e28118 Error response from daemon: conflict: unable to delete 5901cec02503 (cannot be forced) - image is being used by running container 03a56527b683 Error response from daemon: conflict: unable to delete 93ae8cd11560 (cannot be forced) - image is being used by running container f7313ed22d31 Error response from daemon: conflict: unable to delete 714b175e84e8 (cannot be forced) - image is being used by running container 22e7070b53f9 Error response from daemon: conflict: unable to delete 12ad640a1ec0 (cannot be forced) - image is being used by running container 7ee92ef1d188 Adding password for user helk

Pulling helk-elasticsearch ( docker.elastic.co/elasticsearch/elasticsearch:7.1.0)... 7.1.0: Pulling from elasticsearch/elasticsearch Digest: sha256:802b6a299260dbaf21a9c57e3a634491ff788a1ea13a51598d4cd105739509c4 Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.1.0 Pulling helk-kibana (docker.elastic.co/kibana/kibana:7.1.0)... 7.1.0: Pulling from kibana/kibana Digest: sha256:0eb53e2eb9a8846a5a4e700115a4b6e207a8f298128a45cdd28a514f708ec0f3 Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.1.0 Pulling helk-logstash (docker.elastic.co/logstash/logstash:7.1.0)... 7.1.0: Pulling from logstash/logstash Digest: sha256:9258bd2ef10c084a267e470e4fac7b5144e2bd932d628deab4f2c8cc2ff47dd0 Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.1.0 Pulling helk-nginx (cyb3rward0g/helk-nginx:0.0.7)... 0.0.7: Pulling from cyb3rward0g/helk-nginx Digest: sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Status: Downloaded newer image for cyb3rward0g/helk-nginx:0.0.7 Pulling helk-zookeeper (cyb3rward0g/helk-zookeeper:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-zookeeper Digest: sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Status: Downloaded newer image for cyb3rward0g/helk-zookeeper:2.2.0 Pulling helk-kafka-broker (cyb3rward0g/helk-kafka-broker:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-kafka-broker Digest: sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Status: Downloaded newer image for cyb3rward0g/helk-kafka-broker:2.2.0 Pulling helk-ksql-server (confluentinc/cp-ksql-server:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-server Digest: sha256:063add111cc93b1a0118f88b577e31303045d4cc08eb1d21458429f05cba4b02 Status: Downloaded newer image for confluentinc/cp-ksql-server:5.1.3 Pulling helk-ksql-cli (confluentinc/cp-ksql-cli:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-cli Digest: sha256:18c0ccb00fbf87679e16e9e0da600548fcb236a2fd173263b09e89b2d3a42cc3 Status: Downloaded newer image for confluentinc/cp-ksql-cli:5.1.3

Pulling helk-elastalert (cyb3rward0g/helk-elastalert:0.2.4)... 0.2.4: Pulling from cyb3rward0g/helk-elastalert Digest: sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Status: Downloaded newer image for cyb3rward0g/helk-elastalert:0.2.4 Creating helk-elasticsearch ... �[1A�[2K Creating helk-elasticsearch ... �[32mdone�[0m �[1BCreating helk-kibana ... �[1A�[2K Creating helk-kibana ... �[32mdone�[0m �[1BCreating helk-nginx ... Creating helk-logstash ... �[2A�[2K Creating helk-nginx ... �[32mdone�[0m �[2B�[1A�[2K Creating helk-logstash ... �[32mdone�[0m �[1BCreating helk-elastalert ... Creating helk-zookeeper ... �[2A�[2K Creating helk-elastalert ... �[32mdone�[0m �[2B�[1A�[2K Creating helk-zookeeper ... �[32mdone�[0m �[1BCreating helk-kafka-broker ... �[1A�[2K Creating helk-kafka-broker ... �[32mdone�[0m �[1BCreating helk-ksql-server ... �[1A�[2K Creating helk-ksql-server ... �[32mdone�[0m �[1BCreating helk-ksql-cli ... �[1A�[2K Creating helk-ksql-cli ... �[32mdone�[0m �[1Ba268fb4c59d6 d2a8409ecc99 a02a2e4226e7 d87ccecfa214 a268fb4c59d6 d2a8409ecc99 a02a2e4226e7 d87ccecfa214 Untagged: cyb3rward0g/helk-elastalert:0.2.4 Untagged: cyb3rward0g/helk-elastalert@sha256 :dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Deleted: sha256:c7d5d7eed99cce73b7073770dc3c6ecc56a0a447305027ece73e4fabfa0ca51c Deleted: sha256:7ddfedd66acd6b1603c988457aa5cd33417affc4fd9d2e3d768e7ef327bd87e5 Deleted: sha256:19f62ea43ccbd84698f2a9375d08a20d52a02e7f4788843a4d5390cabd3c2472 Deleted: sha256:255396df8ad9053ed73d5a95c12d855e52ad967cd9054676efcfcd85310a1882 Deleted: sha256:df064f144aab0c435fcb883babd6597b724d8f2b73373976f4d3ed964fbbdcee Deleted: sha256:91fe17dde3260a0caf19608ce8e57a69ca0b16ac42e16d489cd08ddb48cb53e9 Deleted: sha256:6872aedee364733b71cd03b5e315ac483f9e6a0bfd61e9a15ca8c3fe7e539936 Untagged: cyb3rward0g/helk-kafka-broker:2.2.0 Untagged: cyb3rward0g/helk-kafka-broker@sha256 :8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Deleted: sha256:c2f7cd82ae1da69dd48c2fc2c26ae855fc8c05efb9e7a8f5d985ebd6c99fddd7 Deleted: sha256:b9059b735d4f7ab0ab47f76fceee5c10d6674739f7d3b2cb89c6cf5db0f92293 Deleted: sha256:52b06347f9930845994d3a549cc9ff99524b3a6ce63ed35b85fa67aa7955ecf8 Deleted: sha256:9b055ea9f21d74cb168ef0277f39de00d15dab2a9af0e7b4c98da0c5c0516c14 Deleted: sha256:7a3378ae3958bfa7bd8bac75f11995dfbb3c277a735e650a621714603a47a7fa Untagged: cyb3rward0g/helk-zookeeper:2.2.0 Untagged: cyb3rward0g/helk-zookeeper@sha256 :a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Deleted: sha256:fa0e4ceb4a83035bd5d80d84cd8827f468821a4178b19df675c135a382c98357 Deleted: sha256:b98cfe820a642bfb0eaaf25bd78f374dc387f78bc19dbdcef67f687d05294c99 Deleted: sha256:abaec1e946404eda0c521c8eb4183450cb341d1098ec3799a5718961dbcd52c5 Deleted: sha256:3a77b7249027c7f216acaaecefcd8241836f172c695cbdf4230491d870153478 Deleted: sha256:a7a618294575059f3ec2fb28a2773cc519e2060479f8af32a78cdbc8e5e2283c Deleted: sha256:f6ea8491f794f0200308a11942febda354d3ccb7dbd8b959e56b8a57c45d8953 Untagged: cyb3rward0g/helk-nginx:0.0.7 Untagged: cyb3rward0g/helk-nginx@sha256 :8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Deleted: sha256:280d044b6719787259c29053d92815e5e11fed4b946d227f84d71ac3c0ec228c Deleted: sha256:90e53da57e1792c0f049cfea1d38f4ac3df3529eefab24ac7e0bea54acbdb426 Deleted: sha256:717f3f4f791b01b6d2926fbb62fe3d5925b31350326ddb68c9f5f636a9c0f64d Deleted: sha256:5d772798c7e2b92a62cf7f07687b67748e277f50e3f313ed0490bfd176408e6b Deleted: sha256:02aaf8287515441c0bfd148e86821a2dc3f711a296044265e42011022ff2664c Deleted: sha256:9a5c1086828b3777b6c1fff86e3437f563be17695ad421397c0a7a0e5a5783b3 Deleted: sha256:762bd884fbde30e58308cd8a21aa2e59dfef049e756015366ca808e5da1a6a59 Deleted: sha256:8961c31eff48193478910b31c7710b7ee57c41e4f12491b6088140cfc119be26 Deleted: sha256:e8b40fc220ff63ad06c8fc48a97bcc06a6e6e344141ca3f5eb0023a01ccfc070 Deleted: sha256:4f0db2f651828b41116784d7299fa130c817808867c092c74de1bd63861fd564 Deleted: sha256:d03c684c7dd711982dddd1ab0e622440c45d27b5d015020e0aef65be0c2e7c9b Deleted: sha256:63eed5473967cb9a5a4c98a5b1d2b6013d368bf88836aab99b5c5178c8f2f51f Deleted: sha256:86267d11f0c14fca869691b9b32bdd610b6ab8d9033d59ee64bdcc2cf0219bce Deleted: sha256:d9a8b3f912eee0b322b86fa0f6888558a468c384611c71178987b20e3a0ebafc Deleted: sha256:4e627d1476f22151f05e5214147d6cc6e03ad79a082f01aca6560aa75c7ade3a Deleted: sha256:757b76a12baba45fcbe76abbdd99723be9d94c12a2ad40354dc49ff5fbe1f5c1 Deleted: sha256:f49017d4d5ce9c0f544c82ed5cbc0672fbcb593be77f954891b22b4d0d4c0a84 Error response from daemon: conflict: unable to delete 785f91d9f484 (cannot be forced) - image is being used by running container 3b8f4b16ee07 Error response from daemon: conflict: unable to delete 5901cec02503 (cannot be forced) - image is being used by running container 26ff8fb43241 Error response from daemon: conflict: unable to delete 93ae8cd11560 (cannot be forced) - image is being used by running container 6e8dd919f3ed Error response from daemon: conflict: unable to delete 714b175e84e8 (cannot be forced) - image is being used by running container 7290a5cd257d Error response from daemon: conflict: unable to delete 12ad640a1ec0 (cannot be forced) - image is being used by running container 3083419626d2 "docker stop" requires at least 1 argument. See 'docker stop --help'.

Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...]

Stop one or more running containers "docker stop" requires at least 1 argument. See 'docker stop --help'.

Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...]

Stop one or more running containers

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cyb3rWard0g/HELK/issues/321?email_source=notifications&email_token=ABQOXYU3R334ROOJX6RV6CLQIOUTLA5CNFSM4ISSF3TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6EY5WA#issuecomment-529108696, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQOXYWDAG5XXEGOQGLEYV3QIOUTLANCNFSM4ISSF3TA .

sugarp0pe commented 5 years ago

No, only trying HELK here, it is fresh install fully updated today

neu5ron commented 5 years ago

which option during install are you using?

Just a note, in the logs you sent it seems the remove script was ran? Thats Ok if it is, I just want to make note of that for troubleshooting later on.

On Sat, Sep 7, 2019 at 10:51 AM sugarp0pe notifications@github.com wrote:

No, only trying HELK here, it is fresh install fully updated today

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cyb3rWard0g/HELK/issues/321?email_source=notifications&email_token=ABQOXYTLJK3X23XU5KUNUWLQIO5WDA5CNFSM4ISSF3TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6E2OQY#issuecomment-529114947, or mute the thread https://github.com/notifications/unsubscribe-auth/ABQOXYW7NFU2XKY3QU34L63QIO5WDANCNFSM4ISSF3TA .

sugarp0pe commented 5 years ago

Running option 2 during helk_install.sh That's right, I've tried to install it twice, use remove script, then manually kill running containers and force remove images after first try

neu5ron commented 5 years ago

lets try to tail the install log the same time the installer is running.
open a terminal/ssh session and run:
tail -F -n1 /var/log/helk-install.log

then open another terminal/ssh session and run the installer with the same options as before (remove all the images beforehand as you had too)

please note and reply where the install script stalls/pauses on a single line for longer than a couple minutes or if it errors out completely. Or just send the entire output of the tail log - once the install script is finished or has has taken too long to complete/finish.

i think there may be some network latency issues either locally or upstream on docker repo side, either way going to work on some more output information for the script and test the same issues again tonight.
also, for note - the remove script needs a little more love too.

neu5ron commented 5 years ago

If you get a chance, please follow the notes/instructions in my last comment.

Just an update on my side (none of this will prevent you from progressing forward): I am still testing adding (more) verbose information for the install process and tweaking the HELK remove script. However, I have not merged/added the changes yet. I have a few more regression tests todo - later on today.

sugarp0pe commented 5 years ago

Hello, Install script hangs on "Starting helk-ksql-cli ... done" and output from tail -F -n1 /var/log/helk-install.log absolutely the same as in the first screenshots in the first issue comment I think there's a problem with elasticsearch container it crashs and restarts every 30 seconds. other containers with install option 2 run properly. Output from docker logs command:

docker logs elasticsearch:7.1.0

[HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:42:20,336+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sever--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,346+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,351+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,352+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,353+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:42:20,353+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-1373145707971171539, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,501+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,505+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,512+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,514+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,514+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,515+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,515+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,516+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,516+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,517+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,521+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,522+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,522+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,525+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,526+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,533+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,534+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,534+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,535+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,537+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,545+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,549+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,551+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,553+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,554+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:42:34,555+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:42:48,258+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:43:05,175+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-1373145707971171539/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-1373145707971171539/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] } [HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:43:16,586+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sever--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,597+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,601+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,602+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,603+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:43:16,604+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-2864629304196106528, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,701+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,701+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,702+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,703+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,704+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,705+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,706+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,708+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,708+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,710+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,711+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,711+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,714+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,715+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,726+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,726+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,727+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,727+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,729+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,730+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,730+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,731+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,732+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,733+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,734+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:43:19,736+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:43:24,169+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:43:36,319+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-2864629304196106528/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-2864629304196106528/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] } [HELK-ES-DOCKER-INSTALLATION-INFO] Setting ES_JAVA_OPTS to -Xms3200m -Xmx3200m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC [HELK-ES-DOCKER-INSTALLATION-INFO] Setting Elastic license to basic [HELK-ES-DOCKER-INSTALLATION-INFO] Running docker-entrypoint script.. {"type": "server", "timestamp": "2019-09-07T12:43:46,183+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/mapper/sever--vg-root)]], net usable_space [860.6gb], net total_space [914.2gb], types [ext4]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,188+0000", "level": "INFO", "component": "o.e.e.NodeEnvironment", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "heap size [3.1gb], compressed ordinary object pointers [true]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,193+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "node name [helk-1], node ID [tQDXBakjTMir3sRmURfvFQ], cluster name [helk-cluster]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,194+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "version[7.1.0], pid[12], build[default/docker/606a173/2019-05-16T00:43:15.323135Z], OS[Linux/4.15.0-60-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/12.0.1/12.0.1+12]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,194+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM home [/usr/share/elasticsearch/jdk]" } {"type": "server", "timestamp": "2019-09-07T12:43:46,195+0000", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-3670512058239285020, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Dio.netty.allocator.type=unpooled, -Des.cgroups.hierarchy.override=/, -Xms3200m, -Xmx3200m, -XX:-UseConcMarkSweepGC, -XX:-UseCMSInitiatingOccupancyOnly, -XX:+UseG1GC, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,189+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [aggs-matrix-stats]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,191+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [analysis-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,192+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-common]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,193+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-geoip]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,194+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [ingest-user-agent]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,198+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-expression]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,201+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-mustache]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,206+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [lang-painless]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,209+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [mapper-extras]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,211+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [parent-join]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,211+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [percolator]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,212+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [rank-eval]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,212+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [reindex]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,213+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [repository-url]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,217+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [transport-netty4]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,218+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ccr]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,223+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-core]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,224+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-deprecation]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-graph]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ilm]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,225+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-logstash]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,229+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-ml]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,230+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-monitoring]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,230+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-rollup]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-security]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-sql]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,231+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "loaded module [x-pack-watcher]" } {"type": "server", "timestamp": "2019-09-07T12:43:49,233+0000", "level": "INFO", "component": "o.e.p.PluginsService", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "no plugins loaded" } {"type": "deprecation", "timestamp": "2019-09-07T12:43:53,630+0000", "level": "WARN", "component": "o.e.d.c.s.Settings", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "[discovery.zen.minimum_master_nodes] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version." } {"type": "server", "timestamp": "2019-09-07T12:44:05,915+0000", "level": "WARN", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "helk-cluster", "node.name": "helk-1", "message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-3670512058239285020/controller_log_12 (No such file or directory)];", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.0.jar:7.1.0]", "Caused by: org.elasticsearch.ElasticsearchException: Failed to create native process factories for Machine Learning", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:433) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more", "Caused by: java.io.FileNotFoundException: /tmp/elasticsearch-3670512058239285020/controller_log_12 (No such file or directory)", "at java.io.FileInputStream.open0(Native Method) ~[?:?]", "at java.io.FileInputStream.open(FileInputStream.java:213) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:155) ~[?:?]", "at java.io.FileInputStream.(FileInputStream.java:110) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:288) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper$PrivilegedInputPipeOpener.run(NamedPipeHelper.java:277) ~[?:?]", "at java.security.AccessController.doPrivileged(AccessController.java:310) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:130) ~[?:?]", "at org.elasticsearch.xpack.ml.utils.NamedPipeHelper.openNamedPipeInputStream(NamedPipeHelper.java:97) ~[?:?]", "at org.elasticsearch.xpack.ml.process.ProcessPipes.connectStreams(ProcessPipes.java:131) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeController.(NativeController.java:61) ~[?:?]", "at org.elasticsearch.xpack.ml.process.NativeControllerHolder.getNativeController(NativeControllerHolder.java:40) ~[?:?]", "at org.elasticsearch.xpack.ml.MachineLearning.createComponents(MachineLearning.java:418) ~[?:?]", "at org.elasticsearch.node.Node.lambda$new$9(Node.java:440) ~[elasticsearch-7.1.0.jar:7.1.0]", "at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) ~[?:?]", "at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654) ~[?:?]", "at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]", "at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]", "at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]", "at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]", "at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]", "at org.elasticsearch.node.Node.(Node.java:443) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.node.Node.(Node.java:252) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.0.jar:7.1.0]", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.0.jar:7.1.0]", "... 6 more"] }

neu5ron commented 5 years ago

Nice catch. Yup that makes sense; and another thing on the radar for the remove script, we need to remove the es_data docker volume (if you dont have any data you need to keep in elasticsearch) - otherwise the trial passwords and such for elasticsearch and kibana need to be identical as last install.

First, run the old remove script (only if you do not have any other docker images/things than HELK stuff) - this is a temporary fix until we can get to “perfecting” the remove script without affecting anybody's existing docker images:
https://github.com/Cyb3rWard0g/HELK/blob/181c851a9ee1fd1f471120f03205048b4a3212f6/docker/helk_remove_containers.sh

Then run : docker volume rm docker_esdata

it may also be called “docker_esdata” but im pretty sure it should be “esdata”

Then run the install script again while simultaneously running the tail command in /var/log/helk-install.log as previously mentioned

Sorry for all of this hassle

sugarp0pe commented 5 years ago

Hello, Today tried to install again, but unfortunately, it is the same result elasticsearch container crashs and restarts all times

Output from helk-install.log

Adding password for user helk Creating network "docker_helk" with driver "bridge" Creating volume "docker_esdata" with local driver Pulling helk-elasticsearch (docker.elastic.co/elasticsearch/elasticsearch:7.1.0)... 7.1.0: Pulling from elasticsearch/elasticsearch Digest: sha256:802b6a299260dbaf21a9c57e3a634491ff788a1ea13a51598d4cd105739509c4 Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.1.0 Pulling helk-kibana (docker.elastic.co/kibana/kibana:7.1.0)... 7.1.0: Pulling from kibana/kibana Digest: sha256:0eb53e2eb9a8846a5a4e700115a4b6e207a8f298128a45cdd28a514f708ec0f3 Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.1.0 Pulling helk-logstash (docker.elastic.co/logstash/logstash:7.1.0)... 7.1.0: Pulling from logstash/logstash Digest: sha256:9258bd2ef10c084a267e470e4fac7b5144e2bd932d628deab4f2c8cc2ff47dd0 Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.1.0 Pulling helk-nginx (cyb3rward0g/helk-nginx:0.0.7)... 0.0.7: Pulling from cyb3rward0g/helk-nginx Digest: sha256:8cdbbd5084f1b7b046af7080a6aea0bf6be0d0e623d990107b9932f7404384e0 Status: Downloaded newer image for cyb3rward0g/helk-nginx:0.0.7 Pulling helk-zookeeper (cyb3rward0g/helk-zookeeper:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-zookeeper Digest: sha256:a4439c74957b0a6c479fe5257f7ce85d6b6ea88b1377323b81cd806cdf23501b Status: Downloaded newer image for cyb3rward0g/helk-zookeeper:2.2.0 Pulling helk-kafka-broker (cyb3rward0g/helk-kafka-broker:2.2.0)... 2.2.0: Pulling from cyb3rward0g/helk-kafka-broker Digest: sha256:8f4caf2b2d7ac98b254e4c3d10d67434d55ee412900520581160b92269873903 Status: Downloaded newer image for cyb3rward0g/helk-kafka-broker:2.2.0 Pulling helk-ksql-server (confluentinc/cp-ksql-server:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-server Digest: sha256:063add111cc93b1a0118f88b577e31303045d4cc08eb1d21458429f05cba4b02 Status: Downloaded newer image for confluentinc/cp-ksql-server:5.1.3 Pulling helk-ksql-cli (confluentinc/cp-ksql-cli:5.1.3)... 5.1.3: Pulling from confluentinc/cp-ksql-cli Digest: sha256:18c0ccb00fbf87679e16e9e0da600548fcb236a2fd173263b09e89b2d3a42cc3 Status: Downloaded newer image for confluentinc/cp-ksql-cli:5.1.3 Pulling helk-elastalert (cyb3rward0g/helk-elastalert:0.2.4)... 0.2.4: Pulling from cyb3rward0g/helk-elastalert Digest: sha256:dc923170fceafc962129baa8cfc733d52c5d02bbb202357baf1b753f9aa6c64d Status: Downloaded newer image for cyb3rward0g/helk-elastalert:0.2.4 Creating helk-elasticsearch ... done Creating helk-elasticsearch ... done Creating helk-kibana ... done Creating helk-kibana ... done Creating helk-nginx ... done Creating helk-logstash ... done Creating helk-nginx ... done Creating helk-logstash ... done Creating helk-elastalert ... done Creating helk-zookeeper ... done Creating helk-zookeeper ... done Creating helk-kafka-broker ... done Creating helk-elastalert ... done Creating helk-kafka-broker ... done Creating helk-ksql-server ... done Creating helk-ksql-server ... done Creating helk-ksql-cli ... done Creating helk-ksql-cli ... done

Relatively remove script I think it would be appropriate to use docker-compose down; docker-compose down --volumes to remove all from of the previous installation, because if try to remove it manually for example: docker volume rm docker_esdata "Error response from daemon: Conflict volume is in use" but no container runing. But in that case, also need to change install script naming schema for docker-compose config files because docker-compose supported filenames can be only: docker-compose.yml and docker-compose.yaml

neu5ron commented 5 years ago

@sugarp0pe. thats a good idea regarding docker remove using the compose file. We could copy the one that gets used during helk setup to the file name that needs used. thanks for recommending that!

regarding the consistent install issues, we honestly can not replicate this.. and apologize for the issues. Lets see if a few other things help us troubleshoot this.

Can you provide the last 500 lines of the elasticsearch container:
docker logs —tail 500 helk-elasticsearch

Also, how much RAM and cpu cores does the OS have?

during the install can you run the following command and let me know if all the RAM gets used ?
watch “free -g”

neu5ron commented 5 years ago

lastly... lets try option 1 for the install. then we can work our way forward after that

Cyb3rWard0g commented 5 years ago

Thank you for your patience @sugarp0pe . I found this related to the

"message": "uncaught exception in thread [main]" , "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[Failed to create native process factories for Machine Learning]; nested: FileNotFoundException[/tmp/elasticsearch-3670512058239285020/controller_log_12 (No such file or directory)];", "a

https://discuss.elastic.co/t/failed-to-start-machine-learning-on-elasticsearch-7-0-0/178216/7

I wonder if it is related to the hardware used to host the build? maybe? Trying to look into all the error messages showing in the Elasticsearch logs.

sugarp0pe commented 5 years ago

@neu5ron, @Cyb3rWard0g great thanks for your help and awesome project! I installed HELK on another machine with same Ubuntu 18.04.3 and it works like a charm. It's definitely a problem related to the hardware like old processor don't supports SSE3 instructions to start ML on elasticsearch. Iam very appreciate for your time to troubleshooting, thank you very much.

Cyb3rWard0g commented 5 years ago

Thank you very much @sugarp0pe for your patience and details you provided for us to be able to help too. Thank you @neu5ron for all the help as well. I appreciate all you are doing for the project too and Im so looking forward to the next release 😉 @sugarp0pe please keep us posted with any issues that you have or questions that you might have about the build. The project is being prepared for a big update soon and we are hoping to get it to BETA by the end of the year, so any feedback is appreciated! ❤️ . I am adding It's definitely a problem related to the hardware like old processor don't supports SSE3 instructions to start ML on elasticsearch to the WIki too. Thank you!

Cyb3rWard0g commented 5 years ago

Added details to WIKI: https://github.com/Cyb3rWard0g/HELK/wiki/Installation

defect-neurology commented 2 years ago

After wasting more time than I would like to admit trying to understand why my very old VM server couldn't seem install HELK, I would like to share the workaround I found.

I found out that by adding the line xpack.ml.enabled: false to the elasticsearch.yml before installing will make it possible to run on older systems as well, see source. You will obviously loose any ML functionality so if that is important you need use newer HW.

According to this site it should be possible to add it to the helk-kibana-*-basic.yml compose files, however I had no luck with doing it this way. I tried several times and the only way I got it to work is by changing the elasticsearch.yml file.

Cyb3rWard0g / HELK

HELK never finishes install #321