search

cuckoosandbox / cuckoo

Cuckoo Sandbox is an automated dynamic malware analysis system
http://www.cuckoosandbox.org
Other
5.55k stars 1.71k forks source link

Cuckoo baseline not working #2752

Open rylore opened 5 years ago

rylore commented 5 years ago

Thanks for creating an issue! But first: did you read our community guidelines? https://cuckoo.sh/docs/introduction/community.html

My issue is:

Baseline command 'cuckoo submit --baseline' is not working, but when I run cuckoo submit --url [yoururl] everything processes fine

My Cuckoo version and operating system are:

Version: 2.0.6 OS: Kali Linux Rolling

This can be reproduced by:

Enter the following into the command line once you have activated the env. cuckoo submit --baseline --url https://[anyurl]

The log, error, files etc can be found at:

It appears that when the VM comes up the agent is unable to communicate due to the 'baseline' flag. I have set up the VM's and cuckoo to run through tor and everything has been working for years. We needed a way to create a baseline of our windows 7 VM and came across the '--baseline' flag.

Main Error Error processing task #660: it appears that the Virtual Machine hasn't been able to contact back to the Cuckoo Host. There could be a few reasons for this, please refer to our documentation on the matter: https://cuckoo.sh/docs/faq/index.html#troubleshooting-vm-network-configuration

Logs [2019-06-13 08:43:18] Processing task #660 2019-06-13 08:43:18,837 [cuckoo.core.scheduler] DEBUG: Processing task #660 [2019-06-13 08:43:18] Starting analysis of BASELINE "none" (task #660, options "") 2019-06-13 08:43:18,847 [cuckoo.core.scheduler] INFO: Starting analysis of BASELINE "none" (task #660, options "") [2019-06-13 08:43:18] Task #660: acquired machine windows7_two (label=windows7_two) 2019-06-13 08:43:18,880 [cuckoo.core.scheduler] INFO: Task #660: acquired machine windows7_two (label=windows7_two) [2019-06-13 08:43:18] Started sniffer with PID 22283 (interface=vboxnet0, host=192.168.56.106) 2019-06-13 08:43:18,893 [cuckoo.auxiliary.sniffer] INFO: Started sniffer with PID 22283 (interface=vboxnet0, host=192.168.56.106) [2019-06-13 08:43:18] Started auxiliary module: Sniffer 2019-06-13 08:43:18,894 [cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer [2019-06-13 08:43:18] Starting vm windows7_two 2019-06-13 08:43:18,914 [cuckoo.machinery.virtualbox] DEBUG: Starting vm windows7_two [2019-06-13 08:43:19] Restoring virtual machine windows7_two to original_12-5-18 2019-06-13 08:43:19,220 [cuckoo.machinery.virtualbox] DEBUG: Restoring virtual machine windows7_two to original_12-5-18 [2019-06-13 08:45:26] Stopped auxiliary module: Sniffer 2019-06-13 08:45:26,261 [cuckoo.core.plugins] DEBUG: Stopped auxiliary module: Sniffer [2019-06-13 08:45:26] Successfully generated memory dump for virtual machine with label windows7_two to path /home/cuckoo/.cuckoo/storage/analyses/660/memory.dmp 2019-06-13 08:45:26,395 [cuckoo.machinery.virtualbox] INFO: Successfully generated memory dump for virtual machine with label windows7_two to path /home/cuckoo/.cuckoo/storage/analyses/660/memory.dmp [2019-06-13 08:45:26] Stopping vm windows7_two 2019-06-13 08:45:26,396 [cuckoo.machinery.virtualbox] DEBUG: Stopping vm windows7_two [2019-06-13 08:45:29] Released database task #660 2019-06-13 08:45:29,037 [cuckoo.core.scheduler] DEBUG: Released database task #660 [2019-06-13 08:45:29] Executed processing module "AnalysisInfo" for task #660 2019-06-13 08:45:29,101 [cuckoo.core.plugins] DEBUG: Executed processing module "AnalysisInfo" for task #660 [2019-06-13 08:45:29] Analysis results folder does not exist at path '/home/cuckoo/.cuckoo/storage/analyses/660/logs'. 2019-06-13 08:45:29,101 [cuckoo.processing.behavior] WARNING: Analysis results folder does not exist at path '/home/cuckoo/.cuckoo/storage/analyses/660/logs'. [2019-06-13 08:45:29] Executed processing module "BehaviorAnalysis" for task #660 2019-06-13 08:45:29,102 [cuckoo.core.plugins] DEBUG: Executed processing module "BehaviorAnalysis" for task #660 [2019-06-13 08:45:29] Executed processing module "Dropped" for task #660 2019-06-13 08:45:29,102 [cuckoo.core.plugins] DEBUG: Executed processing module "Dropped" for task #660 [2019-06-13 08:45:29] Executed processing module "DroppedBuffer" for task #660 2019-06-13 08:45:29,103 [cuckoo.core.plugins] DEBUG: Executed processing module "DroppedBuffer" for task #660 [2019-06-13 08:45:29] VM memory dump not found: to create VM memory dumps you have to enable memory_dump in cuckoo.conf! 2019-06-13 08:45:29,103 [cuckoo.processing.memory] ERROR: VM memory dump not found: to create VM memory dumps you have to enable memory_dump in cuckoo.conf! [2019-06-13 08:45:29] Executed processing module "Memory" for task #660 2019-06-13 08:45:29,103 [cuckoo.core.plugins] DEBUG: Executed processing module "Memory" for task #660 [2019-06-13 08:45:29] Executed processing module "MetaInfo" for task #660 2019-06-13 08:45:29,105 [cuckoo.core.plugins] DEBUG: Executed processing module "MetaInfo" for task #660 [2019-06-13 08:45:29] Executed processing module "ProcessMemory" for task #660 2019-06-13 08:45:29,105 [cuckoo.core.plugins] DEBUG: Executed processing module "ProcessMemory" for task #660 [2019-06-13 08:45:29] Executed processing module "Procmon" for task #660 2019-06-13 08:45:29,106 [cuckoo.core.plugins] DEBUG: Executed processing module "Procmon" for task #660 [2019-06-13 08:45:29] Executed processing module "Screenshots" for task #660 2019-06-13 08:45:29,106 [cuckoo.core.plugins] DEBUG: Executed processing module "Screenshots" for task #660 [2019-06-13 08:45:29] Executed processing module "Static" for task #660 2019-06-13 08:45:29,107 [cuckoo.core.plugins] DEBUG: Executed processing module "Static" for task #660 [2019-06-13 08:45:29] Executed processing module "Strings" for task #660 2019-06-13 08:45:29,107 [cuckoo.core.plugins] DEBUG: Executed processing module "Strings" for task #660 [2019-06-13 08:45:29] Executed processing module "TargetInfo" for task #660 2019-06-13 08:45:29,107 [cuckoo.core.plugins] DEBUG: Executed processing module "TargetInfo" for task #660 [2019-06-13 08:45:29] Executed processing module "Baseline" for task #660 2019-06-13 08:45:29,108 [cuckoo.core.plugins] DEBUG: Executed processing module "Baseline" for task #660 [2019-06-13 08:45:29] Executed processing module "NetworkAnalysis" for task #660 2019-06-13 08:45:29,224 [cuckoo.core.plugins] DEBUG: Executed processing module "NetworkAnalysis" for task #660 [2019-06-13 08:45:29] Executed processing module "VirusTotal" for task #660 2019-06-13 08:45:29,224 [cuckoo.core.plugins] DEBUG: Executed processing module "VirusTotal" for task #660 [2019-06-13 08:45:29] Executed processing module "Extracted" for task #660 2019-06-13 08:45:29,225 [cuckoo.core.plugins] DEBUG: Executed processing module "Extracted" for task #660 [2019-06-13 08:45:29] Executed processing module "TLSMasterSecrets" for task #660 2019-06-13 08:45:29,225 [cuckoo.core.plugins] DEBUG: Executed processing module "TLSMasterSecrets" for task #660 [2019-06-13 08:45:29] Error processing task #660: it appears that the Virtual Machine hasn't been able to contact back to the Cuckoo Host. There could be a few reasons for this, please refer to our documentation on the matter: https://cuckoo.sh/docs/faq/index.html#troubleshooting-vm-network-configuration 2019-06-13 08:45:29,226 [cuckoo.processing.debug] ERROR: Error processing task #660: it appears that the Virtual Machine hasn't been able to contact back to the Cuckoo Host. There could be a few reasons for this, please refer to our documentation on the matter: https://cuckoo.sh/docs/faq/index.html#troubleshooting-vm-network-configuration [2019-06-13 08:45:29] Executed processing module "Debug" for task #660 2019-06-13 08:45:29,259 [cuckoo.core.plugins] DEBUG: Executed processing module "Debug" for task #660 [2019-06-13 08:45:29] Running 549 signatures 2019-06-13 08:45:29,265 [cuckoo.core.plugins] DEBUG: Running 549 signatures [2019-06-13 08:45:29] Analysis matched signature: network_icmp 2019-06-13 08:45:29,440 [cuckoo.core.plugins] DEBUG: Analysis matched signature: network_icmp [2019-06-13 08:45:29] Analysis matched signature: nolookup_communication 2019-06-13 08:45:29,441 [cuckoo.core.plugins] DEBUG: Analysis matched signature: nolookup_communication [2019-06-13 08:45:29] Executed reporting module "ElasticSearch" 2019-06-13 08:45:29,535 [cuckoo.core.plugins] DEBUG: Executed reporting module "ElasticSearch" [2019-06-13 08:45:29] Executed reporting module "JsonDump" 2019-06-13 08:45:29,539 [cuckoo.core.plugins] DEBUG: Executed reporting module "JsonDump" Jun 13 08:45:29 main.c:610 main(): THREAD 0x7efe1ad1e140 [2019-06-13 08:45:30] Executed reporting module "Moloch" 2019-06-13 08:45:30,558 [cuckoo.core.plugins] DEBUG: Executed reporting module "Moloch" [2019-06-13 08:45:30] Executed reporting module "MongoDB" 2019-06-13 08:45:30,576 [cuckoo.core.plugins] DEBUG: Executed reporting module "MongoDB" [2019-06-13 08:45:30] Task #660: reports generation completed 2019-06-13 08:45:30,576 [cuckoo.core.scheduler] INFO: Task #660: reports generation completed [2019-06-13 08:45:30] Task #660: analysis procedure completed 2019-06-13 08:45:30,585 [cuckoo.core.scheduler] INFO: Task #660: analysis procedure completed

rylore commented 5 years ago

Update - I dug through the code for a while and found the following. In order for a 'baseline' to work you first need to create a baseline using the following: 'cuckoo submit --baseline --machine [name of VM]' This can only be done through the command line.

This will then create a task in the DB that the scheduler will pick up that then runs the VM and creates a baseline of it.

Still not sure why it fails to communicate with the back from the VM with the error provided above, but the code is working as designed for the 'cuckoo submit --baseline'.

The baseline is then stored in the $CWD/storage/baseline/[VM name.json]. Next you need to enable the baseline by going into $CWD/conf/processing.conf and change the following: [baseline] enabled = yes.

You can then submit a URL through the UI and once complete the processing task will then kick off the baseline file and should now show up in the report. I'm still not seeing any baseline details in the report, but still digging.

n00btotal commented 4 years ago

I'm trying to figure out how to use the baseline feature as well.. The goal is to (in the GUI) indicate what is part of the standard VM and what is new from the submitted URL/file (i.e standard baseline).

During each baseline submit (cuckoo submit file.exe --memory --baseline --machine cuckoo-win10) I do get the following info: INFO: Starting analysis of BASELINE "none" (task #32, options ""). And later on: Error processing task #2904 : it appears that the Virtual Machine hasn't been able to contact back to the Cuckoo Host. There could be a few reasons for this, please refer to our documentation on the matter: https://cuckoo.sh/docs/faq/index.html#troubleshooting-vm-network-configuration

And when I submit the same file in the Web GUI, there is no comparison with any baseline from what I can tell.

What is baseline supposed to do? I've skimmed through some of the code, but I can't really tell what the end result is supposed to do..?