maliceio / malice

VirusTotal Wanna Be - Now with 100% more Hipster
Apache License 2.0
1.65k stars 265 forks source link

Server Misbehaving.. #45

Closed thebetterjort closed 7 years ago

thebetterjort commented 7 years ago

docker run --net=host --rm -v /var/run/docker.sock:/var/run/docker.sock -vpwd:/malice/samples -e MALICE_VT_API=$MALICE_VT_API malice/engine scan .

2017/04/06 20:29:51 Get http://elastic:9200/: dial tcp: lookup elastic on 127.0.1.1:53: server misbehaving

Docker logs


2017-04-06 20:20:09,356 INFO stopped: nginx (exit status 0)
2017-04-06 20:22:38,014 CRIT Supervisor running as root (no user in config file)
2017-04-06 20:22:38,016 INFO supervisord started with pid 7
2017-04-06 20:22:39,018 INFO spawned: 'nginx' with pid 10
2017-04-06 20:22:39,019 INFO spawned: 'elasticsearch' with pid 11
2017-04-06 20:22:39,019 INFO spawned: 'logstash' with pid 12
2017-04-06 20:22:39,020 INFO spawned: 'kibana' with pid 13
2017-04-06 20:22:39,148 INFO exited: elasticsearch (exit status 1; not expected)
2017-04-06 20:22:40,070 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-06 20:22:40,070 INFO success: logstash entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-04-06 20:22:41,072 INFO spawned: 'elasticsearch' with pid 87
2017-04-06 20:22:41,512 INFO exited: elasticsearch (exit status 1; not expected)
2017-04-06 20:22:44,070 INFO spawned: 'elasticsearch' with pid 120
2017-04-06 20:22:44,070 INFO success: kibana entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
2017-04-06 20:22:44,357 INFO exited: elasticsearch (exit status 1; not expected)
2017-04-06 20:22:47,753 INFO spawned: 'elasticsearch' with pid 148
2017-04-06 20:22:47,945 INFO exited: elasticsearch (exit status 1; not expected)
2017-04-06 20:22:48,471 INFO gave up: elasticsearch entered FATAL state, too many start retries too quickly
2017-04-06 20:24:14,486 WARN received SIGTERM indicating exit request
2017-04-06 20:24:14,486 INFO waiting for nginx, logstash, kibana to die
2017-04-06 20:24:14,491 INFO stopped: kibana (exit status 143)
2017-04-06 20:24:17,632 INFO waiting for nginx, logstash to die
2017-04-06 20:24:20,634 INFO waiting for nginx, logstash to die
2017-04-06 20:24:21,251 INFO stopped: logstash (exit status 0)
2017-04-06 20:24:22,290 INFO stopped: nginx (exit status 0)
blacktop commented 7 years ago

can you tell me a little about the machine you are running on?

also run this: $ docker exec malice-elk head -n30 /var/log/elasticsearch.stdout.log

blacktop commented 7 years ago

My guess is you have less that 2GB of RAM, which elasticsearch wants to run.

thebetterjort commented 7 years ago

Strange machine has 4GB physical memory.

docker exec malice-elk head -n30 /var/log/elasticsearch.stdout.log

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1798569984 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmp/hs_err_pid20.log
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1798569984 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmp/hs_err_pid57.log
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1798569984 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmp/hs_err_pid89.log
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1798569984 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmp/hs_err_pid154.log
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1798569984 bytes for committing reserved memory.
# Can not save log file, dump to screen..
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1798569984 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
blacktop commented 7 years ago

hmmm maybe you need to do the following: https://github.com/maliceio/malice#known-issues-warning ?

blacktop commented 7 years ago

I am adding checks in malice to detect elasticsearch failures and warn the user of low RAM etc.

You can also see how much RAM docker has by running docker info | grep Mem