Unable to process files

[berssutti]

About accounts on capesandbox.com

• Issues isn't the way to ask for account activation. Ping capesandbox in Twitter with your username

This is open source and you are getting free support so be friendly!

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [ X ] I am running the latest version
• [ X ] I did read the README!
• [ X ] I checked the documentation and found no answer
• [ X ] I checked to make sure that this issue has not already been filed
• [ X ] I'm reporting the issue to the correct repository (for multi-repository projects)
• [ X ] I have read and checked all configs (with all optional parts)
• [ X ] I'm aware that configs now are in custom/conf/

Expected Behavior

Having the files processed and the results of processing in the storage/analysis folder or in the ip address I configured.

Current Behavior

When I run the application with poetry run python3 cuckoo.py and then submit a analysis with sudo -u cape poetry run python3 submit.py /path/to/binary/test.exe nothing happens. When I tried to reload the address where I configured the results to be(192.168.56.1:2042), it gives me a WARNING: ResultServer did not have a task for IP 192.168.56.1. After several minute I got a critical error CRITICAL: CuckooCriticalError: Please update your configuration. Unable to shut 'Win7' down or find the machine in its proper state: Timeout hit while for machine Win7 to change status.

cuckoo.conf

[cuckoo]

# Which category of tasks do you want to analyze?
categories = static, pcap, url, file

# If turned on, Cuckoo will delete the original file after its analysis
# has been completed.
delete_original = off

# Archives are not deleted by default, as it extracts and "original file" become extracted file
delete_archive = on

# If turned on, Cuckoo will delete the copy of the original file in the
# local binaries repository after the analysis has finished. (On *nix this
# will also invalidate the file called "binary" in each analysis directory,
# as this is a symlink.)
delete_bin_copy = off

# Specify the name of the machinery module to use, this module will
# define the interaction between Cuckoo and your virtualization software
# of choice.
machinery = virtualbox

# Enable creation of memory dump of the analysis machine before shutting
# down. Even if turned off, this functionality can also be enabled at
# submission. Currently available for: VirtualBox and libvirt modules (KVM).
memory_dump = off

# When the timeout of an analysis is hit, the VM is just killed by default.
# For some long-running setups it might be interesting to terminate the
# moinitored processes before killing the VM so that connections are closed.
terminate_processes = off

# Enable automatically re-schedule of "broken" tasks each startup.
# Each task found in status "processing" is re-queued for analysis.
reschedule = off

# Fail "unserviceable" tasks as they are queued.
# Any task found that will never be analyzed based on the available analysis machines
# will have its status set to "failed".
fail_unserviceable = on

# Limit the amount of analysis jobs a Cuckoo process goes through.
# This can be used together with a watchdog to mitigate risk of memory leaks.
max_analysis_count = 0

# Limit the number of concurrently executing analysis machines.
# This may be useful on systems with limited resources.
# Set to 0 to disable any limits.
max_machines_count = 10

# Limit the amount of VMs that are allowed to start in parallel. Generally
# speaking starting the VMs is one of the more CPU intensive parts of the
# actual analysis. This option tries to avoid maxing out the CPU completely.
max_vmstartup_count = 5

# Minimum amount of free space (in MB) available before starting a new task.
# This tries to avoid failing an analysis because the reports can't be written
# due out-of-diskspace errors. Setting this value to 0 disables the check.
# (Note: this feature is currently not supported under Windows.)
freespace = 0
# Process tasks, but not reach out of memory
freespace_processing = 15000

# Temporary directory containing the files uploaded through Cuckoo interfaces
# (web.py, api.py, Django web interface).
tmppath = /tmp

# Delta in days from current time to set the guest clocks to for file analyses
# A negative value sets the clock back, a positive value sets it forward.
# The default of 0 disables this option
# Note that this can still be overridden by the per-analysis clock setting
# and it is not performed by default for URL analysis as it will generally
# result in SSL errors
daydelta = 0

# Path to the unix socket for running root commands.
rooter = /tmp/cuckoo-rooter

# Enable if you want to see a DEBUG log periodically containing backlog of pending tasks, locked vs unlocked machines.
# NOTE: Enabling this feature adds 4 database calls every 10 seconds.
periodic_log = off

# Max filename length for submissions, before truncation. 196 is arbitrary.
max_len = 196

# If it is greater than this, call truncate the filename further for sanitizing purposes.
# Length truncated to is controlled by sanitize_to_len.
#
# This is to prevent long filenames such as files named by hash.
sanitize_len = 32
sanitize_to_len = 24

[resultserver]
# The Result Server is used to receive in real time the behavioral logs
# produced by the analyzer.
# Specify the IP address of the host. The analysis machines should be able
# to contact the host through such address, so make sure it's valid.
# NOTE: if you set resultserver IP to 0.0.0.0 you have to set the option
# `resultserver_ip` for all your virtual machines in machinery configuration.
ip = 192.168.56.1

# Specify a port number to bind the result server on.
port = 2042

# Force the port chosen above, don't try another one (we can select another
# port dynamically if we can not bind this one, but that is not an option
# in some setups)
force_port = yes

pool_size = 0

# Should the server write the legacy CSV format?
# (if you have any custom processing on those, switch this on)
store_csvs = off

# Maximum size of uploaded files from VM (screenshots, dropped files, log)
# The value is expressed in bytes, by default 100MB.
upload_max_size = 100000000

# To enable trimming of huge binaries go to -> web.conf -> general -> enable_trim
# Prevent upload of files that passes upload_max_size?
do_upload_max_size = no

[processing]
# Set the maximum size of analyses generated files to process. This is used
# to avoid the processing of big files which may take a lot of processing
# time. The value is expressed in bytes, by default 200MB.
analysis_size_limit = 200000000

# The number of calls per process to process. 0 switches the limit off.
# 10000 api calls should be processed in less than 2 minutes
analysis_call_limit = 0

# Enable or disable DNS lookups.
resolve_dns = on

# Enable or disable reverse DNS lookups
# This information currently is not displayed in the web interface
reverse_dns = off

# Use ram to boost processing speed. You will need more than 20GB of RAM for this feature.
# Please read "performance" section in the documentation.
ram_boost = no

# Enable PCAP sorting, needed for the connection content view in the web interface.
sort_pcap = on

[database]
# Specify the database connection string.
# Examples, see documentation for more:
# sqlite:///foo.db
# postgresql://foo:bar@localhost:5432/mydatabase
# mysql://foo:bar@localhost/mydatabase
# If empty, default is a SQLite in db/cuckoo.db.
# SQLite doens't support database upgrades!
# For production we strongly suggest go with PostgreSQL
connection = postgresql://cape:SuperPuperSecret@localhost:5432/cape

# Database connection timeout in seconds.
# If empty, default is set to 60 seconds.
timeout =

[timeouts]
# Set the default analysis timeout expressed in seconds. This value will be
# used to define after how many seconds the analysis will terminate unless
# otherwise specified at submission.
default = 200

# Set the critical timeout expressed in (relative!) seconds. It will be added
# to the default timeout above and after this timeout is hit
# Cuckoo will consider the analysis failed and it will shutdown the machine
# no matter what. When this happens the analysis results will most likely
# be lost.
critical = 60

# Maximum time to wait for virtual machine status change. For example when
# shutting down a vm. Default is 300 seconds.
vm_state = 300

[tmpfs]
# only if you using volatility to speedup IO
# mkdir -p /mnt/tmpfs
# mount -t tmpfs -o size=50g ramfs /mnt/tmpfs
# chown cape:cape /mnt/tmpfs
#
# vim /etc/fstab
# tmpfs       /mnt/tmpfs tmpfs   nodev,nosuid,noexec,nodiratime,size=50g   0 0
#
# Add crontab with
# @reboot chown cape:cape /mnt/tmpfs -R
enabled = off
path = /mnt/tmpfs/
# in mb
freespace = 2000

virtualbox.conf

[virtualbox]
# Specify which VirtualBox mode you want to run your machines on.
# Can be "gui", "sdl" or "headless". Refer to VirtualBox's official
# documentation to understand the differences.
mode = gui

# Path to the local installation of the VBoxManage utility.
path = /usr/bin/VBoxManage

interface = vboxnet0
# Specify a comma-separated list of available machines to be used. For each
# specified ID you have to define a dedicated section containing the details
# on the respective machine. (E.g. cuckoo1,cuckoo2,cuckoo3)
machines = Win7

[Win7]
# Specify the label name of the current machine as specified in your
# VirtualBox configuration.
label = Win7

# Specify the operating system platform used by current machine
# [windows/darwin/linux].
platform = windows

# Specify the IP address of the current virtual machine. Make sure that the
# IP address is valid and that the host machine is able to reach it. If not,
# the analysis will fail.
ip = 192.168.56.2

# Set the machine architecture
# Required to auto select proper machine architecture for sample
# x64 or x86
arch = x64

# (Optional) Specify tags to display
# Tags may be used to specify on which guest machines a sample should be run
# tags = windows_xp_sp3,acrobat_reader_6

# (Optional) Specify the snapshot name to use. If you do not specify a snapshot
# name, the VirtualBox MachineManager will use the current snapshot.
# Example (Snapshot1 is the snapshot name):
#snapshot = 

# (Optional) Specify the name of the network interface that should be used
# when dumping network traffic from this machine with tcpdump. If specified,
# overrides the default interface specified in auxiliary.conf
# Example (vboxnet0 is the interface name):
# interface = vboxnet0

# (Optional) Specify the IP of the Result Server, as your virtual machine sees it.
# The Result Server will always bind to the address and port specified in cuckoo.conf,
# however you could set up your virtual network to use NAT/PAT, so you can specify here
# the IP address for the Result Server as your machine sees it. If you don't specify an
# address here, the machine will use the default value from cuckoo.conf.
# NOTE: if you set this option you have to set result server IP to 0.0.0.0 in cuckoo.conf.
# Example:
# resultserver_ip = 192.168.56.1

# (Optional) Specify the port for the Result Server, as your virtual machine sees it.
# The Result Server will always bind to the address and port specified in cuckoo.conf,
# however you could set up your virtual network to use NAT/PAT, so you can specify here
# the port for the Result Server as your machine sees it. If you don't specify a port
# here, the machine will use the default value from cuckoo.conf.
# Example:
# resultserver_port = 2042

# (Optional) Specify a memory profile to be used by volatility for this
# virtual machine. This will override the guest_profile variable in
# memory.conf which solves the problem of having multiple types of VM's
# and properly determining which profile to use. Examples below:
#mem_profile = WinXPSP2x86
#mem_profile = Win7SP0x86
#mem_profile = Win7SP1x64

Failure Information (for bugs)

Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.

Steps to Reproduce

Please provide detailed steps for reproducing the issue.

1. step 1
2. step 2
3. you get it...

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions. Operating system version, bitness, installed software versions, test sample details/hash/binary (if applicable).

Question	Answer
Git commit	Type $ git log \| head -n1 to find out
OS version	Ubuntu 16.04, Windows 10, macOS 10.12.3

Failure Logs

Please include any relevant log snippets or files here.

[doomedraven]

hey did you ensure that agent is running inside of the VM? the confs are correct, we don't use vbox so idk if that is some vbox problems

[doomedraven]

https://capev2.readthedocs.io/en/latest/installation/guest/agent.html?highlight=curl#installing-the-agent

[berssutti]

Hi, thank you for your reply. I had the agent.py running already, here's a screenshot of the agent output in the vm ip adress:

One thing I forgot to mention, is that every time I execute the poetry run python3 cuckoo.py command, it print a message saying that the alias "CAPE_SERVICE" is already defined. I didn't understand what that message means, and I couldn't say if it is connected to the error I am having.

[doomedraven]

if your vm is running, then it sound like vbox changed some of the apis maybe to get state etc, if you want to dig into that here is the code for vbox https://github.com/kevoreilly/CAPEv2/blob/master/modules/machinery/virtualbox.py

we don't use vbox so you will need to handle that by yourself or wait till someone who uses that tries to find the problem

[mjdepuy]

Sorry, random person who just happened to install CAPE with vbox haha. I ran into a similar warning message about the VM state being unable to be found. Mine was a networking issue, so I would double check that all the networking stuff is in order.

1. When you type ip addr are you seeing vboxnet0?
2. Did you assign your guest VM to vboxnet0?
3. Did you completely disable the Windows firewall? <-- This is what tripped me up, public firewall was still on.
4. Is the Ubuntu firewall off?
5. Can you send pings from guest -> 192.168.56.1 and host -> guest IP?
6. Tried assigning a different IP address? (If you don't pay attention to the Virtual network adapter settings, the adapter only supports 192.168.56.101 - 192.168.56.254, so that could also be an issue; this one also got me)

Once I got all that sorted, I was able to start analyses. Sorry if it isn't that much help, but since I was also plagued by this error message, I figured I would throw some troubleshooting steps out :)

[doomedraven]

thank you @mjdepuy. any extra input is useful for area where devs can't help. lets see if that helps him

[berssutti]

Thanks for the replies @doomedraven @mjdepuy. I'll give it a try now and let you know how it goes ASAP.

[berssutti]

Updates

I was unable to make it work with VBox, so I ended up installing KVM and setting up Cape to run with it. Although I am no longer encountering the same errors as before, I am still having trouble in obtaining results.

Issue

After running Cape for several minutes, I receive the following error message: ERROR: Machine win10: the guest initialization hit the critical timeout, analysis aborted . It appears that the error is occurring in this session of the code (guest.py):

I attempted to debug with Python console to determine whether the connection is happening or not, and apparently it is.

I've run out of ideas on what I could do to solve this.

---

Another error is occurring with the database when I attempt to submit an .exe file. Which is strange because this issue does not occur with a .dll file. Sadly, I don't have a clue what's causing this error. Error: adding task to database

[berssutti]

Updates

I have successfully resolved the error with the guest initialization. However, although my code can now process .dll files(I am still unable to submit .exe files), the logs of execution are not being sent to the result server. When I refreshed the result server IP, I received the message: WARNING: ResultServer did not have a task for IP 192.168.122.1.

While debugging, I discovered that this issue is occurring because the code is attempting to retrieve the value of the host's IP address, while the task_id value is stored in the guest IP. I am not sure if this is intentional or not, and I cannot locate where this function is called. I would appreciate any help in resolving this issue.

[doomedraven]

well it works just fine for us, so you should have something bad in configuration of kvm or vm, not sure