VMs working standalone, cuckoo analysis doesn't produce network data

[jeffhammett]

I followed Seanthegeek's instructions here https://infosecspeakeasy.org/t/howto-build-a-cuckoo-sandbox/27 and setup Cuckoo-modified. Aside from changing the ip in kvm.conf and changing the interface in auxiliary.conf I followed the above directions exactly.

If I boot the vm (without Cuckoo) and run tcpdump on virbr0 I am able to see network requests (but the VM has no internet connection as expected). If I run "sudo torstart 192.168.100.101" the VM gets internet from Tor and I see the network traffic in tcpdump.

When I submit URLs or files to Cuckoo via the web interface I always get: "Error: The analysis hit the critical timeout, terminating." The pcap is empty. If I submit a file there will be details about the file (VT results, etc.) but it doesn't seem like the file has actually run. If I submit a URL the report is essentially empty. If I check the option for Tor when submitting the file I get ARP traffic in the pcap file, but nothing else.

Since the VM seems to work ok on its own, I'm starting my troubleshooting with Cuckoo.

If I watch the VM while running a Cuckoo analysis in the KVM console, I don't see anything happening, but looking at task manager it appears to be running the proper processes (iexplore.exe for URLs, winword.exe for Doc files, etc.)

If I manually open Internet Explorer while watching the VM in the KVM console while Cuckoo is running an analysis I see the network traffic for it loading my homepage in the results, but the VM seems to freeze up before I can do anything in the IE window.

There are also two pythonw.exe processes running while running a Cuckoo analysis, but only one when running the VM standalone, is this normal?

Here is my auxiliary.conf, cuckoo.conf, kvm.conf, processing.conf and reporting.conf

Any help would be much appreciated.

analysis@sandbox:/opt/cuckoo/conf$ cat auxiliary.conf 
[sniffer]
# Enable or disable the use of an external sniffer (tcpdump) [yes/no].
enabled = yes

# Specify the path to your local installation of tcpdump. Make sure this
# path is correct.
tcpdump = /usr/sbin/tcpdump

# Specify the network interface name on which tcpdump should monitor the
# traffic. Make sure the interface is active.
interface = virbr0

# Specify a Berkeley packet filter to pass to tcpdump.
# bpf = not arp

[tor]
# Enable or disable the use of Tor transparent proxying
# Note that this is a global enable/disable. It is still required that
# you specifically enable Tor for each analyzed sample from the
# web interface.
#
# Please note that in order to implement this functionality securely
# without any additional privilege on the part of Cuckoo, the below
# scripts should simply pass the IP address of the VM used for analysis 
# to a daemon running as root, which can run the
# iptables rules itself. For a working example, see
# https://github.com/seanthegeek/routetor
enabled = yes

# Specify the path to a binary or script that will initiate the firewall
# rules to redirect traffic to the Tor transparent proxy.  The file
# will be executed with the argument of the static IP of the VM used
# for analysis.
torstart = /usr/sbin/torstart

# Specify the path to a binary or script that will eliminate the firewall
# rules used to redirect traffic to the Tor transparent proxy.  The file
# will be executed with the argument of the static IP of the VM used
# for analysis.
torstop = /usr/sbin/torstop

[gateways]
#RTR1 = 192.168.1.254
#RTR2 = 192.168.1.1
#INETSIM = 192.168.1.2

[virustotaldl]
# adds an option in the web interface to upload samples via VirusTotal
# downloads for a comma-separated list of MD5/SHA1/SHA256 hashes
enabled = no
# note that unlike the VirusTotal processing module, the key required
# here is a Private API key, not a Public API key
#dlprivkey = SomeKeyWithDLAccess
# alternatively if you have VirusTotal Intelligence access, you can
# supply your Public API key below.  Only one of these keys may be
# uncommented at a time.
dlintelkey = SomeKeyWithDLAccess
dlpath = /tmp/

# Web UI settings

[display_browser_martians]
enabled = no

[display_office_martians]
enabled = no

[display_shrike]
enabled = no

[expanded_dashboard]
# displays package, custom field, malfamily, clamav, PCAP link, and extended suricata results
enabled = no

[display_et_portal]
enabled = no

[display_pt_portal]
enabled = no
analysis@sandbox:/opt/cuckoo/conf$ cat cuckoo.conf 
[cuckoo]
# If turned on, Cuckoo will delete the original file after its analysis
# has been completed.
delete_original = off

# If turned on, Cuckoo will delete the copy of the original file in the
# local binaries repository after the analysis has finished. (On *nix this
# will also invalidate the file called "binary" in each analysis directory,
# as this is a symlink.)
delete_bin_copy = off

# Specify the name of the machinery module to use, this module will
# define the interaction between Cuckoo and your virtualization software
# of choice.
machinery = kvm

# Enable creation of memory dump of the analysis machine before shutting
# down. Even if turned off, this functionality can also be enabled at
# submission. Currently available for: VirtualBox and libvirt modules (KVM).
memory_dump = off

# When the timeout of an analysis is hit, the VM is just killed by default.
# For some long-running setups it might be interesting to terminate the
# moinitored processes before killing the VM so that connections are closed.
terminate_processes = off

# Enable automatically re-schedule of "broken" tasks each startup.
# Each task found in status "processing" is re-queued for analysis.
reschedule = off

# Enable processing of results within the main cuckoo process.
# This is the default behavior but can be switched off for setups that
#  require high stability and process the results in a separate task.
process_results = on

# Limit the amount of analysis jobs a Cuckoo process goes through.
# This can be used together with a watchdog to mitigate risk of memory leaks.
max_analysis_count = 0

# Limit the number of concurrently executing analysis machines.
# This may be useful on systems with limited resources.
# Set to 0 to disable any limits.
max_machines_count = 0

# Limit the amount of VMs that are allowed to start in parallel. Generally
# speaking starting the VMs is one of the more CPU intensive parts of the
# actual analysis. This option tries to avoid maxing out the CPU completely.
max_vmstartup_count = 10

# Minimum amount of free space (in MB) available before starting a new task. 
# This tries to avoid failing an analysis because the reports can't be written 
# due out-of-diskspace errors. Setting this value to 0 disables the check.
# (Note: this feature is currently not supported under Windows.)
freespace = 64

# Temporary directory containing the files uploaded through Cuckoo interfaces
# (web.py, api.py, Django web interface).
tmppath = /tmp

# Delta in days from current time to set the guest clocks to for file analyses
# Since some malware families are increasingly using time expirations, it
# is best for automation purposes to set the clock back a number of months.
# By default we now set the clock back 4 months.  To disable this change,
# set daydelta to 0
# Note that this can still be overridden by the per-analysis clock setting
# and it is not performed by default for URL analysis as it will generally
# result in SSL errors
daydelta = -120

[resultserver]
# The Result Server is used to receive in real time the behavioral logs
# produced by the analyzer.
# Specify the IP address of the host. The analysis machines should be able
# to contact the host through such address, so make sure it's valid.
# NOTE: if you set resultserver IP to 0.0.0.0 you have to set the option
# `resultserver_ip` for all your virtual machines in machinery configuration.
ip = 192.168.100.1

# Specify a port number to bind the result server on.
port = 2042

# Should the server write the legacy CSV format?
# (if you have any custom processing on those, switch this on)
store_csvs = off

# Maximum size of uploaded files from VM (screenshots, dropped files, log)
# The value is expressed in bytes, by default 10Mb.
upload_max_size = 10485760

[processing]
# Set the maximum size of analyses generated files to process. This is used
# to avoid the processing of big files which may take a lot of processing
# time. The value is expressed in bytes, by default 100Mb.
analysis_size_limit = 104857600

# The number of calls per process to process. 0 switches the limit off.
#10000 api calls should be processed in less than 2 minutes
analysis_call_limit = 0

# Enable or disable DNS lookups.
resolve_dns = on

# Enable or disable reverse DNS lookups
# This information currently is not displayed in the web interface
reverse_dns = off

# Use ram to boost processing speed. You will need more than 20GB of RAM for this feature.
# Please read "performance" section in the documentation.
ram_boost = off

# Enable PCAP sorting, needed for the connection content view in the web interface.
sort_pcap = on

[database]
# Specify the database connection string.
# Examples, see documentation for more:
# sqlite:///foo.db
# postgresql://foo:bar@localhost:5432/mydatabase
# mysql://foo:bar@localhost/mydatabase
# If empty, default is a SQLite in db/cuckoo.db.
connection = postgresql://cuckoo:redacted@localhost:5432/cuckoo

# Database connection timeout in seconds.
# If empty, default is set to 60 seconds.
timeout =

[timeouts]
# Set the default analysis timeout expressed in seconds. This value will be
# used to define after how many seconds the analysis will terminate unless
# otherwise specified at submission.
default = 240

# Set the critical timeout expressed in (relative!) seconds. It will be added
# to the default timeout above and after this timeout is hit
# Cuckoo will consider the analysis failed and it will shutdown the machine
# no matter what. When this happens the analysis results will most likely
# be lost.
critical = 60

# Maximum time to wait for virtual machine status change. For example when
# shutting down a vm. Default is 300 seconds.
vm_state = 300
analysis@sandbox:/opt/cuckoo/conf$ cat kvm.conf 
[kvm]
# Specify a comma-separated list of available machines to be used. For each
# specified ID you have to define a dedicated section containing the details
# on the respective machine. (E.g. cuckoo1,cuckoo2,cuckoo3)
machines = sandbox-win7-01

[sandbox-win7-01]
# Specify the label name of the current machine as specified in your
# libvirt configuration.
label = sandbox-win7-01

# Specify the operating system platform used by current machine
# [windows/darwin/linux].
platform = windows

# Specify the IP address of the current virtual machine. Make sure that the
# IP address is valid and that the host machine is able to reach it. If not,
# the analysis will fail. You may want to configure your network settings in
# /etc/libvirt/<hypervisor>/networks/
ip = 192.168.100.101

# (Optional) Specify the snapshot name to use. If you do not specify a snapshot
# name, the KVM MachineManager will use the current snapshot.
# Example (Snapshot1 is the snapshot name):
snapshot = running

# (Optional) Specify the name of the network interface that should be used
# when dumping network traffic from this machine with tcpdump. If specified,
# overrides the default interface specified in auxiliary.conf
# Example (virbr0 is the interface name):
#interface = virbr0

# (Optional) Specify the IP of the Result Server, as your virtual machine sees it.
# The Result Server will always bind to the address and port specified in cuckoo.conf,
# however you could set up your virtual network to use NAT/PAT, so you can specify here
# the IP address for the Result Server as your machine sees it. If you don't specify an
# address here, the machine will use the default value from cuckoo.conf.
# NOTE: if you set this option you have to set result server IP to 0.0.0.0 in cuckoo.conf.
# Example:
# resultserver_ip = 192.168.122.101

# (Optional) Specify the port for the Result Server, as your virtual machine sees it.
# The Result Server will always bind to the address and port specified in cuckoo.conf,
# however you could set up your virtual network to use NAT/PAT, so you can specify here
# the port for the Result Server as your machine sees it. If you don't specify a port
# here, the machine will use the default value from cuckoo.conf.
# Example:
# resultserver_port = 2042

# (Optional) Set your own tags. These are comma separated and help to identify
# specific VMs. You can run samples on VMs with tag you require.
# Note that the 64_bit tag is currently special.  For submitted 64-bit PE files,
# the 64_bit tag will automatically be added, forcing them to be run on a 64-bit
# VM.  For this reason, make sure all 64-bit VMs have the 64_bit tag.
tags = windows_7,64_bit,acrobat_reader_10,java_6_flash_15

# (Optional) Specify a memory profile to be used by volatility for this
# virtual machine. This will override the guest_profile variable in
# memory.conf which solves the problem of having multiple types of VM's
# and properly determining which profile to use. Examples below:
#mem_profile = WinXPSP2x86
#mem_profile = Win7SP0x86
mem_profile = Win7SP1x64
analysis@sandbox:/opt/cuckoo/conf$ cat processing.conf 
# Enable or disable the available processing modules [on/off].
# If you add a custom processing module to your Cuckoo setup, you have to add
# a dedicated entry in this file, or it won't be executed.
# You can also add additional options under the section of your module and
# they will be available in your Python class.

[analysisinfo]
enabled = yes

[decompression]
enabled = yes

[behavior]
enabled = yes

[debug]
enabled = yes

[dropped]
enabled = yes
# Amount of text to carve from plaintext files (bytes)
buffer = 8192

[memory]
enabled = no

[usage]
enabled = no

[network]
enabled = yes
# DNS whitelisting to ignore domains/IPs configured in network.py
# This should be disabled when utilizing InetSim/Remnux as we end up resolving
# the IP from fakedns which would then remove all domains associated with that
# resolved IP
dnswhitelist = no

[procmemory]
enabled = yes
strings = yes

[static]
enabled = yes
# Enable a WHOIS lookup for the target domain of a URL analyses
whois = yes
# If you want to use the Procyon Java decompiler, set the following to the path of its .jar
# and make sure Java 7 is installed
# Procyon is developed by Mike Strobel and is available at https://bitbucket.org/mstrobel/procyon/
# In testing, it generally seems to produce the best Java decompilation
procyon_path = /home/cuckoo/procyon-decompiler-0.5.30.jar

[strings]
enabled = yes
nullterminated_only = yes
minchars = 5

[targetinfo]
enabled = yes

[virustotal]
enabled = yes
timeout = 60
# Add your VirusTotal API key here. The default API key, kindly provided
# by the VirusTotal team, should enable you with a sufficient throughput
# and while being shared with all our users, it shouldn't affect your use.
key = a0283a2c3d55728300d064874239b5346fb991317e8449fe43c902879d758088
do_file_lookup = yes
do_url_lookup = yes
urlscrub = (^http:\/\/serw\.clicksor\.com\/redir\.php\?url=|&InjectedParam=.+$)

[suricata]
# Notes on getting this to work:
# Install Suricata 2.1 beta
# $ sudo add-apt-repository ppa:oisf/suricata-beta
# $ sudo apt-get update
# $ sudo apt-get install libhtp1 suricata
# Create /etc/suricata/rules/cuckoo.rules:
# echo "alert http any any -> any any (msg:\"FILE store all\"; filestore; noalert; sid:15; rev:1;)"  | sudo tee /etc/suricata/rules/cuckoo.rules
# Edit /etc/suricata/suricata.yaml
# Ensure the eve log is enabled
# You can disable the fast and unified2-alert logs to save space
# Enable file-store, set force-md5 to yes
# Enable file-log
# Add " - cuckoo.rules" to the list under "rules-files:"
# Under "reassembly:", which is under "stream:", set depth to 0 (without any measurement unit)
# Set request-body-limit and response-body-limit to 0 (without any measurement unit), under "default-config:"
# You may also need the following line under "stream:", but test it without it first
#   async-oneside: yes
# Set EXTERNAL_NET to "any"
# You can use the etupdate script to install and update the Emerging Threats ruleset
# https://github.com/seanthegeek/etupdate

##GlobalSettings
enabled = yes
#Runmode "cli" or "socket"
runmode = cli
#Outputfiles
# if evelog is specified, it will be used instead of the per-protocol log files
evelog = eve.json

# per-protocol log files
#
#alertlog = alert.json
#httplog = http.json
#tlslog = tls.json
#sshlog = ssh.json
#dnslog = dns.json

fileslog = files-json.log
filesdir = files
# Amount of text to carve from plaintext files (bytes)
buffer = 8192
#Used for creating an archive of extracted files
7zbin = /usr/bin/7z
zippass = infected
##Runmode "cli" options
bin = /usr/bin/suricata
conf = /etc/suricata/suricata-cuckoo.yaml
##Runmode "socket" Options
pylib_dir = /usr/lib/python2.7/dist-packages/suricatasc/
socket_file = /var/run/suricata/suricata-command.socket

[cif]
enabled = no
# url of CIF server
url = https://your-cif-server.com/api
# CIF API key
key = your-api-key-here
# time to wait for server to respond, in seconds
timeout = 60
# minimum confidence level of returned results:
#25=not confident, 50=automated, 75=somewhat confident, 85=very confident, 95=certain
# defaults to 85
confidence = 85
# don't log queries by default, set to 'no' to log queries
nolog = yes
# max number of results per query
per_lookup_limit = 20
# max number of queries per analysis
per_analysis_limit = 200
analysis@sandbox:/opt/cuckoo/conf$ cat reporting.conf 
# Enable or disable the available reporting modules [on/off].
# If you add a custom reporting module to your Cuckoo setup, you have to add
# a dedicated entry in this file, or it won't be executed.
# You can also add additional options under the section of your module and
# they will be available in your Python class.

[jsondump]
enabled = yes
# use the c-optimized JSON encoder, requires fitting entire JSON results in memory
ram_boost = yes
indent = 4
encoding = latin-1

[reporthtml]
# required for the WSGI interface
enabled = no

[reporthtmlsummary]
# much smaller, faster report generation, omits API logs and is non-interactive
enabled = yes

[reportpdf]
# Note that this requires reporthtmlsummary to be enabled above as well
enabled = yes

[mmdef]
enabled = no

[maec41]
enabled = no
mode = overview
processtree = true
output_handles = false
static = true
strings = true
virustotal = true
deduplicate = true

[mongodb]
enabled = yes
host = 127.0.0.1
port = 27017
db = cuckoo
# Automatically delete large dict values that exceed mongos 16MB limitation
# Note: This only deletes dict keys from data stored in MongoDB. You would
# still get the full dataset if you parsed the results dict in another
# reporting module or from the jsondump module.
fix_large_docs = yes

# Use ElasticSearch as the "database" which powers Django.
# NOTE: If this is enabled, MongoDB should not be enabled.
[elasticsearchdb]
enabled = no
host = 127.0.0.1
port = 9200
# The report data is indexed in the form of {{index-yyyy.mm.dd}}
# so the below index configuration option is actually an index 'prefix'.
index = cuckoo

[retention]
enabled = no
# run at most once every this many hours (unless reporting.conf is modified)
run_every = 6
# The amount of days old a task needs to be before deleting data
# Set a value to no to never delete it
memory = 14
procmemory = 62
pcap = 62
sortedpcap = 14
bsonlogs = 62
dropped = 62
screencaps = 62
reports = 62
mongo = 731
elastic = no
malheur = no

[syslog]
enabled = no
# IP of your syslog server/listener
host = x.x.x.x
# Port of your syslog server/listener
port = 514
# Protocol to send data over
protocol = tcp
# Store a logfile? [in reports directory]
logfile = yes
# if yes, what logname? [Default: syslog.txt]
logname = syslog.log

[moloch]
enabled = no
base = https://172.18.100.105:8005/
node = cuckoo3
capture = /data/moloch/bin/moloch-capture
captureconf = /data/moloch/etc/config.ini
user = admin
pass = admin
realm = Moloch

[resubmitexe]
enabled = yes
resublimit = 5

[malheur]
enabled = yes
maxsimilar = 20

[compression]
enabled = yes
zipmemdump = yes
zipmemstrings = yes
zipprocdump = yes
zipprocstrings = yes

[misp]
enabled = no
apikey =
url =
# by default 5 threads
threads =

[jeffhammett]

I got this working. It ended up not being an issue with Cuckoo directly. When Cuckoo restored the running VM in KVM the VM was freezing. I changed the cache mode on the virtual hard drive to "none" and it is working now.