spender-sandbox / cuckoo-modified

Modified edition of cuckoo
395 stars 178 forks source link

cuckoo1: the guest initialization hit the critical timeout, analysis aborted. #503

Open me0ne0 opened 6 years ago

me0ne0 commented 6 years ago

i am continuously getting the above mentioned error. I tired to increase the time limit upto 3600 seconds with memory dump off but its still the same error. Also i tried different sampls, all result in the same manner. why is it happening?

doomedraven commented 6 years ago

that is bcz your network or agent is misconfigured, follow the documentation

me0ne0 commented 6 years ago

the network and agent are communicating

curl 10.10.10.13:8000

Error response

Error response

Error code 501.

Message: Unsupported method ('GET').

Error code explanation: 501 = Server does not support this operation.

me0ne0 commented 6 years ago

i am able to ping guest VM at 10.10.10.13

64 bytes from 10.10.10.13: icmp_seq=7716 ttl=128 time=0.273 ms 64 bytes from 10.10.10.13: icmp_seq=7717 ttl=128 time=0.413 ms 64 bytes from 10.10.10.13: icmp_seq=7718 ttl=128 time=0.323 ms 64 bytes from 10.10.10.13: icmp_seq=7719 ttl=128 time=0.409 ms 64 bytes from 10.10.10.13: icmp_seq=7720 ttl=128 time=0.398 ms 64 bytes from 10.10.10.13: icmp_seq=7721 ttl=128 time=0.312 ms

doomedraven commented 6 years ago

post, logs, and versions of everything

me0ne0 commented 6 years ago

hi, sorry for replying late i was out of town for a few days.

Following is the log of the task that i tried to run

2018-04-16 21:22:10,700 [lib.cuckoo.core.scheduler] INFO: Using "virtualbox" machine manager with max_analysis_count=0, max_machines_count=0, and max_vmstartup_count=10 2018-04-16 21:22:13,313 [lib.cuckoo.core.scheduler] INFO: Loaded 1 machine/s 2018-04-16 21:22:13,329 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks. 2018-04-16 21:25:43,726 [lib.cuckoo.core.scheduler] INFO: Task #11: Starting analysis of FILE '/tmp/cuckoo-tmp/upload_fdbO1E/0.exe' 2018-04-16 21:25:43,736 [lib.cuckoo.core.scheduler] INFO: Task #11: File already exists at '/home/mxn/cuckoo-modified/storage/binaries/09a1c17ac55cde962b4f3bcd61140d752d86362296ee74736000a6a647c73d8c' 2018-04-16 21:25:43,779 [lib.cuckoo.core.scheduler] INFO: Task #11: acquired machine cuckoo1 (label=win764-1C) 2018-04-16 21:25:54,428 [modules.auxiliary.sniffer] INFO: Started sniffer with PID 12162 (interface=vboxnet1, host=10.10.10.13 interface = vboxnet1 mem_profile = Win7SP1x64, dump path=/home/mxn/cuckoo-modified/storage/analyses/11/dump.pcap) 2018-04-16 21:25:54,482 [lib.cuckoo.core.guest] INFO: Starting analysis on guest (id=cuckoo1, ip=10.10.10.13 interface = vboxnet1 mem_profile = Win7SP1x64) 2018-04-16 21:31:54,961 [lib.cuckoo.core.scheduler] ERROR: cuckoo1: the guest initialization hit the critical timeout, analysis aborted. 2018-04-16 21:31:58,146 [modules.processing.behavior] WARNING: Analysis results folder does not exist at path "/home/mxn/cuckoo-modified/storage/analyses/11/logs". 2018-04-16 21:31:58,151 [lib.cuckoo.core.plugins] ERROR: Failed to run the processing module "Dropped": Traceback (most recent call last): File "/home/mxn/cuckoo-modified/lib/cuckoo/core/plugins.py", line 197, in process data = current.run() File "/home/mxn/cuckoo-modified/modules/processing/dropped.py", line 26, in run file_names = os.listdir(self.dropped_path) OSError: [Errno 2] No such file or directory: '/home/mxn/cuckoo-modified/storage/analyses/11/files' 2018-04-16 21:31:58,156 [modules.processing.network] WARNING: The PCAP file does not exist at path "/home/mxn/cuckoo-modified/storage/analyses/11/dump.pcap". 2018-04-16 21:32:04,735 [elasticsearch] WARNING: PUT http://127.0.0.1:9200/cuckoo-2018-04-16/analysis/11 [status:N/A request:0.001s] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 897, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 859, in send self.connect() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcb97527e90>: Failed to establish a new connection: [Errno 111] Connection refused 2018-04-16 21:32:04,738 [elasticsearch] WARNING: PUT http://127.0.0.1:9200/cuckoo-2018-04-16/analysis/11 [status:N/A request:0.000s] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 897, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 859, in send self.connect() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcb97527710>: Failed to establish a new connection: [Errno 111] Connection refused 2018-04-16 21:32:04,739 [elasticsearch] WARNING: PUT http://127.0.0.1:9200/cuckoo-2018-04-16/analysis/11 [status:N/A request:0.000s] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 897, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 859, in send self.connect() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcb97527e90>: Failed to establish a new connection: [Errno 111] Connection refused 2018-04-16 21:32:04,740 [elasticsearch] WARNING: PUT http://127.0.0.1:9200/cuckoo-2018-04-16/analysis/11 [status:N/A request:0.000s] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 897, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 859, in send self.connect() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fcb97527710>: Failed to establish a new connection: [Errno 111] Connection refused 2018-04-16 21:32:04,741 [lib.cuckoo.core.plugins] ERROR: Failed to run the reporting module "ElasticsearchDB": Traceback (most recent call last): File "/home/mxn/cuckoo-modified/lib/cuckoo/core/plugins.py", line 631, in process current.run(self.results) File "/home/mxn/cuckoo-modified/modules/reporting/elasticsearchdb.py", line 143, in run self.es.index(index=self.index_name, doc_type="analysis", id=results["info"]["id"], body=report) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 76, in _wrapped return func(*args, params=params, **kwargs) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/init.py", line 319, in index _make_path(index, doc_type, id), params=params, body=body) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 314, in perform_request status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 175, in perform_request raise ConnectionError('N/A', str(e), e) ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fcb97527710>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fcb97527710>: Failed to establish a new connection: [Errno 111] Connection refused) 2018-04-16 21:32:04,742 [lib.cuckoo.core.scheduler] INFO: Task #11: reports generation completed (path=/home/mxn/cuckoo-modified/storage/analyses/11) 2018-04-16 21:32:04,770 [lib.cuckoo.core.scheduler] INFO: Task #11: analysis procedure completed

me0ne0 commented 6 years ago

Its my understanding that for all the versions of cuckoo guest VM's should be in snapshot resume mode. As all guest VM's should be started by Cuckoo itself as mentioned by you. But in snapshot resume mode neither curl works nor guest VM IP's are pingable. Is that could be problem ?

doomedraven commented 6 years ago

yes they should be in running mode.

ERROR: cuckoo1: the guest initialization hit the critical timeout, analysis aborted. that means what cuckoo can't communicate with agent in vm.

host=10.10.10.13, can you post the configs, it might be some small misconfiguration

me0ne0 commented 6 years ago

let me post the VB guest configuration.

Please note as it was interfering with the cuckoosandbox so i changed the IP scheme to 192.168.58.1/24 as 192.168.58.1 the host ip and 192.168.58.11 the guest VM IP and network interface to vboxnet2 this means 10.10.10.13 = 192.168.58.11

CUCKOO.CONF [cuckoo]

If turned on, Cuckoo will delete the original file after its analysis

has been completed.

delete_original = off

If turned on, Cuckoo will delete the copy of the original file in the

local binaries repository after the analysis has finished. (On *nix this

will also invalidate the file called "binary" in each analysis directory,

as this is a symlink.)

delete_bin_copy = off

Specify the name of the machinery module to use, this module will

define the interaction between Cuckoo and your virtualization software

of choice.

machinery = virtualbox

Enable creation of memory dump of the analysis machine before shutting

down. Even if turned off, this functionality can also be enabled at

submission. Currently available for: VirtualBox and libvirt modules (KVM).

memory_dump = off

When the timeout of an analysis is hit, the VM is just killed by default.

For some long-running setups it might be interesting to terminate the

moinitored processes before killing the VM so that connections are closed.

terminate_processes = off

Enable automatically re-schedule of "broken" tasks each startup.

Each task found in status "processing" is re-queued for analysis.

reschedule = off

Enable processing of results within the main cuckoo process.

This is the default behavior but can be switched off for setups that

require high stability and process the results in a separate task.

process_results = on

Limit the amount of analysis jobs a Cuckoo process goes through.

This can be used together with a watchdog to mitigate risk of memory leaks.

max_analysis_count = 0

Limit the number of concurrently executing analysis machines.

This may be useful on systems with limited resources.

Set to 0 to disable any limits.

max_machines_count = 0

Limit the amount of VMs that are allowed to start in parallel. Generally

speaking starting the VMs is one of the more CPU intensive parts of the

actual analysis. This option tries to avoid maxing out the CPU completely.

max_vmstartup_count = 10

Minimum amount of free space (in MB) available before starting a new task.

This tries to avoid failing an analysis because the reports can't be written

due out-of-diskspace errors. Setting this value to 0 disables the check.

(Note: this feature is currently not supported under Windows.)

freespace = 64

Temporary directory containing the files uploaded through Cuckoo interfaces

(web.py, api.py, Django web interface).

tmppath = /tmp

Delta in days from current time to set the guest clocks to for file analyses

A negative value sets the clock back, a positive value sets it forward.

The default of 0 disables this option

Note that this can still be overridden by the per-analysis clock setting

and it is not performed by default for URL analysis as it will generally

result in SSL errors

daydelta = 0

[resultserver]

The Result Server is used to receive in real time the behavioral logs

produced by the analyzer.

Specify the IP address of the host. The analysis machines should be able

to contact the host through such address, so make sure it's valid.

NOTE: if you set resultserver IP to 0.0.0.0 you have to set the option

resultserver_ip for all your virtual machines in machinery configuration.

ip = 192.168.58.1

Specify a port number to bind the result server on.

port = 2043

Should the server write the legacy CSV format?

(if you have any custom processing on those, switch this on)

store_csvs = on

Maximum size of uploaded files from VM (screenshots, dropped files, log)

The value is expressed in bytes, by default 10Mb.

upload_max_size = 10485760

[processing]

Set the maximum size of analyses generated files to process. This is used

to avoid the processing of big files which may take a lot of processing

time. The value is expressed in bytes, by default 100Mb.

analysis_size_limit = 104857600

The number of calls per process to process. 0 switches the limit off.

10000 api calls should be processed in less than 2 minutes

analysis_call_limit = 0

Enable or disable DNS lookups.

resolve_dns = on

Enable or disable reverse DNS lookups

This information currently is not displayed in the web interface

reverse_dns = off

Use ram to boost processing speed. You will need more than 20GB of RAM for this feature.

Please read "performance" section in the documentation.

ram_boost = off

Enable PCAP sorting, needed for the connection content view in the web interface.

sort_pcap = on

[database]

Specify the database connection string.

Examples, see documentation for more:

sqlite:///foo.db

postgresql://foo:bar@localhost:5432/mydatabase

mysql://foo:bar@localhost/mydatabase

If empty, default is a SQLite in db/cuckoo.db.

connection = postgresql://postgres:meen@localhost:5432/cuckoopost

Database connection timeout in seconds.

If empty, default is set to 60 seconds.

timeout =

[timeouts]

Set the default analysis timeout expressed in seconds. This value will be

used to define after how many seconds the analysis will terminate unless

otherwise specified at submission.

default = 120

Set the critical timeout expressed in (relative!) seconds. It will be added

to the default timeout above and after this timeout is hit

Cuckoo will consider the analysis failed and it will shutdown the machine

no matter what. When this happens the analysis results will most likely

be lost.

critical = 60

Maximum time to wait for virtual machine status change. For example when

shutting down a vm. Default is 300 seconds.

vm_state = 300

VIRTUAL BOX.CONF


[virtualbox]

Specify which VirtualBox mode you want to run your machines on.

Can be "gui", "sdl" or "headless". Refer to VirtualBox's official

documentation to understand the differences.

mode = headless

Path to the local installation of the VBoxManage utility.

path = /usr/bin/VBoxManage

Specify a comma-separated list of available machines to be used. For each

specified ID you have to define a dedicated section containing the details

on the respective machine. (E.g. cuckoo1,cuckoo2,cuckoo3)

machines = cuckoo1

[cuckoo1]

Specify the label name of the current machine as specified in your

VirtualBox configuration.

label = win764-1C

Specify the operating system platform used by current machine

[windows/darwin/linux].

platform = windows

Specify the IP address of the current virtual machine. Make sure that the

IP address is valid and that the host machine is able to reach it. If not,

the analysis will fail.

ip = 192.168.58.11

(Optional) Specify the snapshot name to use. If you do not specify a snapshot

name, the VirtualBox MachineManager will use the current snapshot.

Example (Snapshot1 is the snapshot name):

snapshot = snapshot1

(Optional) Specify the name of the network interface that should be used

when dumping network traffic from this machine with tcpdump. If specified,

overrides the default interface specified in auxiliary.conf

Example (vboxnet0 is the interface name):

interface = vboxnet2

(Optional) Specify the IP of the Result Server, as your virtual machine sees it.

The Result Server will always bind to the address and port specified in cuckoo.conf,

however you could set up your virtual network to use NAT/PAT, so you can specify here

the IP address for the Result Server as your machine sees it. If you don't specify an

address here, the machine will use the default value from cuckoo.conf.

NOTE: if you set this option you have to set result server IP to 0.0.0.0 in cuckoo.conf.

Example:

resultserver_ip = 192.168.56.1

(Optional) Specify the port for the Result Server, as your virtual machine sees it.

The Result Server will always bind to the address and port specified in cuckoo.conf,

however you could set up your virtual network to use NAT/PAT, so you can specify here

the port for the Result Server as your machine sees it. If you don't specify a port

here, the machine will use the default value from cuckoo.conf.

Example:

resultserver_port = 2042

(Optional) Set your own tags. These are comma separated and help to identify

specific VMs. You can run samples on VMs with tag you require.

Note that the 64_bit tag is currently special. For submitted 64-bit PE files,

the 64_bit tag will automatically be added, forcing them to be run on a 64-bit

VM. For this reason, make sure all 64-bit VMs have the 64_bit tag.

tags = windows_xp_sp3,32_bit,acrobat_reader_6

(Optional) Specify a memory profile to be used by volatility for this

virtual machine. This will override the guest_profile variable in

memory.conf which solves the problem of having multiple types of VM's

and properly determining which profile to use. Examples below:

mem_profile = WinXPSP2x86

mem_profile = Win7SP0x86

mem_profile = Win7SP1x64

doomedraven commented 6 years ago

conf looks good, any custom iptables?

me0ne0 commented 6 years ago

yes i had custom IP tables for the previous subnets 192.168.56.1 and 10.10.10.1. But i had no rules assigned for 192.168.58.1. following is the result.

Chain INPUT (policy ACCEPT) target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:bootps ACCEPT tcp -- anywhere anywhere tcp dpt:bootps ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:bootps ACCEPT tcp -- anywhere anywhere tcp dpt:bootps ACCEPT udp -- anywhere anywhere udp dpt:domain ACCEPT tcp -- anywhere anywhere tcp dpt:domain ACCEPT udp -- anywhere anywhere udp dpt:bootps ACCEPT tcp -- anywhere anywhere tcp dpt:bootps ufw-before-logging-input all -- anywhere anywhere
ufw-before-input all -- anywhere anywhere
ufw-after-input all -- anywhere anywhere
ufw-after-logging-input all -- anywhere anywhere
ufw-reject-input all -- anywhere anywhere
ufw-track-input all -- anywhere anywhere

Chain FORWARD (policy ACCEPT) target prot opt source destination
ACCEPT all -- anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable REJECT all -- anywhere anywhere reject-with icmp-port-unreachable ACCEPT all -- anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable REJECT all -- anywhere anywhere reject-with icmp-port-unreachable ACCEPT all -- anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable REJECT all -- anywhere anywhere reject-with icmp-port-unreachable ACCEPT all -- 10.0.0.0/8 anywhere ctstate NEW ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT all -- 10.0.0.0/8 anywhere ctstate NEW ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT all -- 192.168.56.0/24 anywhere ctstate NEW ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ACCEPT all -- 10.0.0.0/8 anywhere ctstate NEW ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED ufw-before-logging-forward all -- anywhere anywhere
ufw-before-forward all -- anywhere anywhere
ufw-after-forward all -- anywhere anywhere
ufw-after-logging-forward all -- anywhere anywhere
ufw-reject-forward all -- anywhere anywhere
ufw-track-forward all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT) target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:bootpc ACCEPT udp -- anywhere anywhere udp dpt:bootpc ACCEPT udp -- anywhere anywhere udp dpt:bootpc ufw-before-logging-output all -- anywhere anywhere
ufw-before-output all -- anywhere anywhere
ufw-after-output all -- anywhere anywhere
ufw-after-logging-output all -- anywhere anywhere
ufw-reject-output all -- anywhere anywhere
ufw-track-output all -- anywhere anywhere

Chain ufw-after-forward (1 references) target prot opt source destination

Chain ufw-after-input (1 references) target prot opt source destination

Chain ufw-after-logging-forward (1 references) target prot opt source destination

Chain ufw-after-logging-input (1 references) target prot opt source destination

Chain ufw-after-logging-output (1 references) target prot opt source destination

Chain ufw-after-output (1 references) target prot opt source destination

Chain ufw-before-forward (1 references) target prot opt source destination

Chain ufw-before-input (1 references) target prot opt source destination

Chain ufw-before-logging-forward (1 references) target prot opt source destination

Chain ufw-before-logging-input (1 references) target prot opt source destination

Chain ufw-before-logging-output (1 references) target prot opt source destination

Chain ufw-before-output (1 references) target prot opt source destination

Chain ufw-reject-forward (1 references) target prot opt source destination

Chain ufw-reject-input (1 references) target prot opt source destination

Chain ufw-reject-output (1 references) target prot opt source destination

Chain ufw-track-forward (1 references) target prot opt source destination

Chain ufw-track-input (1 references) target prot opt source destination

Chain ufw-track-output (1 references) target prot opt source destination

me0ne0 commented 6 years ago

@doomedraven hello, any update on this that could be useful?

doomedraven commented 6 years ago

sudo iptables -F && sudo iptables -F -t nat && sudo ufw disable try to do this and restart cuckoo.py

me0ne0 commented 6 years ago

not working, still same error cuckoo1: the guest initialization hit the critical timeout, analysis aborted.

Can you do me a favor? If your cuckoo is working fine, can u pack ur conf folder (all files) and email me so i can cross ref it with my config files? or i can do it vice versa and you can cross ref them, which ever option suits you.

me0ne0 commented 6 years ago

here is the complete log

=======================================

2018-04-20 00:59:18,219 [lib.cuckoo.core.scheduler] INFO: Using "virtualbox" machine manager with max_analysis_count=0, max_machines_count=0, and max_vmstartup_count=10 2018-04-20 00:59:19,478 [lib.cuckoo.core.scheduler] INFO: Loaded 1 machine/s 2018-04-20 00:59:19,493 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks. 2018-04-20 01:00:11,440 [lib.cuckoo.core.scheduler] INFO: Task #20: Starting analysis of FILE '/tmp/cuckoo-tmp/upload_WMMnGB/2d.exe' 2018-04-20 01:00:11,461 [lib.cuckoo.core.scheduler] INFO: Task #20: File already exists at '/home/mxn/cuckoo-modified/storage/binaries/8b1177549a1f4a0e47acd8ec77bf670ee18efb9f2c18747e460bd8924d5a2024' 2018-04-20 01:00:11,505 [lib.cuckoo.core.scheduler] INFO: Task #20: acquired machine cuckoo1 (label=win764-1C) 2018-04-20 01:00:21,915 [modules.auxiliary.sniffer] INFO: Started sniffer with PID 13583 (interface=vboxnet2, host=192.168.58.11 snapshot = snapshot1 interface = vboxnet2 mem_profile = Win7SP1x64, dump path=/home/mxn/cuckoo-modified/storage/analyses/20/dump.pcap) 2018-04-20 01:00:21,964 [lib.cuckoo.core.guest] INFO: Starting analysis on guest (id=cuckoo1, ip=192.168.58.11 snapshot = snapshot1 interface = vboxnet2 mem_profile = Win7SP1x64) 2018-04-20 01:03:22,223 [lib.cuckoo.core.scheduler] ERROR: cuckoo1: the guest initialization hit the critical timeout, analysis aborted. 2018-04-20 01:03:24,800 [modules.processing.behavior] WARNING: Analysis results folder does not exist at path "/home/mxn/cuckoo-modified/storage/analyses/20/logs". 2018-04-20 01:03:24,806 [lib.cuckoo.core.plugins] ERROR: Failed to run the processing module "Dropped": Traceback (most recent call last): File "/home/mxn/cuckoo-modified/lib/cuckoo/core/plugins.py", line 197, in process data = current.run() File "/home/mxn/cuckoo-modified/modules/processing/dropped.py", line 26, in run file_names = os.listdir(self.dropped_path) OSError: [Errno 2] No such file or directory: '/home/mxn/cuckoo-modified/storage/analyses/20/files' 2018-04-20 01:03:24,811 [modules.processing.network] WARNING: The PCAP file does not exist at path "/home/mxn/cuckoo-modified/storage/analyses/20/dump.pcap". 2018-04-20 01:03:25,493 [lib.cuckoo.common.objects] WARNING: failed to scan file with clamav Error 2 connecting /var/run/clamav/clamd.ctl. No such file or directory.

(wkhtmltopdf:13735): IBUS-WARNING : Unable to connect to ibus: Could not connect: Connection refused 2018-04-20 01:03:29,162 [elasticsearch] WARNING: PUT http://127.0.0.1:9200/cuckoo-2018-04-20/analysis/20 [status:N/A request:0.001s] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 897, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 859, in send self.connect() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fad458ab410>: Failed to establish a new connection: [Errno 111] Connection refused 2018-04-20 01:03:29,165 [elasticsearch] WARNING: PUT http://127.0.0.1:9200/cuckoo-2018-04-20/analysis/20 [status:N/A request:0.000s] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 897, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 859, in send self.connect() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fad458ab950>: Failed to establish a new connection: [Errno 111] Connection refused 2018-04-20 01:03:29,166 [elasticsearch] WARNING: PUT http://127.0.0.1:9200/cuckoo-2018-04-20/analysis/20 [status:N/A request:0.000s] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 897, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 859, in send self.connect() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fad458ab510>: Failed to establish a new connection: [Errno 111] Connection refused 2018-04-20 01:03:29,167 [elasticsearch] WARNING: PUT http://127.0.0.1:9200/cuckoo-2018-04-20/analysis/20 [status:N/A request:0.000s] Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 166, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, kw) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen _stacktrace=sys.exc_info()[2]) File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 357, in _make_request conn.request(method, url, *httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1057, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 897, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 859, in send self.connect() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fad458ab410>: Failed to establish a new connection: [Errno 111] Connection refused 2018-04-20 01:03:29,167 [lib.cuckoo.core.plugins] ERROR: Failed to run the reporting module "ElasticsearchDB": Traceback (most recent call last): File "/home/mxn/cuckoo-modified/lib/cuckoo/core/plugins.py", line 631, in process current.run(self.results) File "/home/mxn/cuckoo-modified/modules/reporting/elasticsearchdb.py", line 143, in run self.es.index(index=self.index_name, doc_type="analysis", id=results["info"]["id"], body=report) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 76, in _wrapped return func(args, params=params, **kwargs) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/init.py", line 319, in index _make_path(index, doc_type, id), params=params, body=body) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 314, in perform_request status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout) File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 175, in perform_request raise ConnectionError('N/A', str(e), e) ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7fad458ab410>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7fad458ab410>: Failed to establish a new connection: [Errno 111] Connection refused) 2018-04-20 01:03:29,169 [lib.cuckoo.core.scheduler] INFO: Task #20: reports generation completed (path=/home/mxn/cuckoo-modified/storage/analyses/20) 2018-04-20 01:03:29,636 [lib.cuckoo.core.scheduler] INFO: Task #20: analysis procedure completed

=============================================================

doomedraven commented 6 years ago

im not using vbox so that won't be useful for you, but it looks like network error or network configuration on your side

me0ne0 commented 6 years ago

i cannot think of any other way to resolve this issue. How about if you send me [api auxiliary cuckoo processing reporting memory conf files]?

doomedraven commented 6 years ago

nop, they don't have anything to see here, that is network related issue

me0ne0 commented 6 years ago

agent.py or agent.pyw what should be used?

doomedraven commented 6 years ago

that is the same, just without window -> documentation

me0ne0 commented 6 years ago

@doomedraven i tried almost everything. I re-ran agent.py from cuckoo modified and from original cuckoosandbox I re-ran agent.pyw from cuckoo modified and from original cuckoosandbox

curl output to both is OK. i am able to telnet both on 8000 port. I can ping guest 192.168.58.11 from host 192.168.58.1 and vice versa.

Tried the snapshot restore, and guest power off method.

All of these resulted in guest VM being up as could be seen in Gui but the debug log continuously shows

2018-04-20 02:45:54,116 [lib.cuckoo.core.resultserver] DEBUG: ResultServer running on 192.168.58.1:2043. 2018-04-20 02:45:54,119 [lib.cuckoo.core.scheduler] INFO: Using "virtualbox" machine manager with max_analysis_count=0, max_machines_count=0, and max_vmstartup_count=10 2018-04-20 02:45:54,402 [modules.machinery.virtualbox] DEBUG: Getting status for win764-1C 2018-04-20 02:45:54,565 [modules.machinery.virtualbox] DEBUG: Machine win764-1C status saved 2018-04-20 02:45:54,595 [modules.machinery.virtualbox] DEBUG: Stopping vm win764-1C 2018-04-20 02:45:54,595 [modules.machinery.virtualbox] DEBUG: Getting status for win764-1C 2018-04-20 02:45:54,743 [modules.machinery.virtualbox] DEBUG: Machine win764-1C status saved 2018-04-20 02:45:55,785 [modules.machinery.virtualbox] DEBUG: VBoxManage exited with error powering off the machine 2018-04-20 02:45:55,786 [modules.machinery.virtualbox] DEBUG: Getting status for win764-1C 2018-04-20 02:45:55,960 [modules.machinery.virtualbox] DEBUG: Machine win764-1C status saved 2018-04-20 02:45:56,631 [lib.cuckoo.core.scheduler] INFO: Loaded 1 machine/s 2018-04-20 02:45:56,646 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks. 2018-04-20 02:47:35,195 [lib.cuckoo.core.scheduler] DEBUG: Task #24: Processing task 2018-04-20 02:47:35,199 [lib.cuckoo.core.scheduler] INFO: Task #24: Starting analysis of FILE '/tmp/cuckoo-tmp/upload_SJzyIq/8bc503de-3038-11e8-8625-80e65024849a.exe' 2018-04-20 02:47:35,402 [lib.cuckoo.core.scheduler] INFO: Task #24: acquired machine cuckoo1 (label=win764-1C) 2018-04-20 02:47:35,524 [modules.machinery.virtualbox] DEBUG: Starting vm win764-1C 2018-04-20 02:47:35,524 [modules.machinery.virtualbox] DEBUG: Getting status for win764-1C 2018-04-20 02:47:35,688 [modules.machinery.virtualbox] DEBUG: Machine win764-1C status saved 2018-04-20 02:47:35,797 [modules.machinery.virtualbox] DEBUG: Using current snapshot for virtual machine win764-1C 2018-04-20 02:47:38,252 [modules.machinery.virtualbox] DEBUG: Getting status for win764-1C 2018-04-20 02:47:38,403 [modules.machinery.virtualbox] DEBUG: Machine win764-1C status saved 2018-04-20 02:47:45,217 [modules.machinery.virtualbox] DEBUG: Getting status for win764-1C 2018-04-20 02:47:45,413 [modules.machinery.virtualbox] DEBUG: Machine win764-1C status running 2018-04-20 02:47:45,531 [modules.auxiliary.sniffer] INFO: Started sniffer with PID 17859 (interface=vboxnet2, host=192.168.58.11 snapshot = snapshot1 interface = vboxnet2 mem_profile = Win7SP1x64, dump path=/home/mxn/cuckoo-modified/storage/analyses/24/dump.pcap) 2018-04-20 02:47:45,532 [lib.cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer 2018-04-20 02:47:45,608 [lib.cuckoo.core.guest] INFO: Starting analysis on guest (id=cuckoo1, ip=192.168.58.11 snapshot = snapshot1 interface = vboxnet2 mem_profile = Win7SP1x64) 2018-04-20 02:47:45,609 [lib.cuckoo.core.guest] DEBUG: cuckoo1: waiting for status 0x0001 2018-04-20 02:47:45,610 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:46,612 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:47,615 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:48,616 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:49,618 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:50,620 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:51,622 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:52,623 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:53,625 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:54,627 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:55,629 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:47:56,631 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:48:01,166 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:48:02,168 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:48:03,171 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:48:04,173 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:48:05,175 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:48:06,177 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:48:07,178 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet 2018-04-20 02:48:08,180 [lib.cuckoo.core.guest] DEBUG: cuckoo1: not ready yet

which i understood and as mentioned by you is that agent is not communicating or there is network error.

firewall is disabled on both, bit defender, uac, updates are off on guest vm. even i have disabled user login.

how the hell could i get rid of this error ? its been 3 weeks now, i am unable to find its solution..

can it be the issue of virtualbox as hypervisor?

doomedraven commented 6 years ago

could be, vbox doing weird iptables stuff nowdays

my suggestion, start cuckoo task for example url analysis with timeout 1000 and without user interaction, and once vm is started, test curl vm:8000, ping is useless for this purpose so you will be able to investigate a bit the network and the rest

me0ne0 commented 6 years ago

still useless

screenshot from 2018-04-20 23-49-48

doomedraven commented 6 years ago

the error not make any sense, idk what is wrong here

me0ne0 commented 6 years ago

i now configured KVM and result is the same. It appears that there is some kind of problem from guest side.

doomedraven commented 6 years ago

idk

me0ne0 commented 6 years ago

can you tag anyone else who can help me out here? :/

doomedraven commented 6 years ago

all interested person receiving email about all issues so they can decide if they want to help or not

piero1974 commented 5 years ago

puoi taggare qualcun altro che può aiutarmi qui? : /

did you resolve it? i've the same problems and i can't resolve it ! Help!