cuckoosandbox / cuckoo

Cuckoo Sandbox is an automated dynamic malware analysis system
http://www.cuckoosandbox.org
Other
5.54k stars 1.7k forks source link

Empty Analysis: AttributeError: Config instance has no attribute 'file_type' #1849

Open hollus opened 7 years ago

hollus commented 7 years ago

Hi All,

I am having no results being returned when a URL analysis is finished in Cuckoo. I am running OSX on my host box and also running OSX in my virtual machine in VirtualBox. Below is the analyzer log I received from the web GUI:

2017-09-13 17:29:01,780 [root] DEBUG: Starting analyzer from /tmpKoSNUL
2017-09-13 17:29:01,791 [root] DEBUG: Storing results at: /tmp/xUOUgI
2017-09-13 17:29:01,791 [root] ERROR: Traceback (most recent call last):
  File "/tmpKoSNUL/analyzer.py", line 192, in <module>
    success = analyzer.run()
  File "/tmpKoSNUL/analyzer.py", line 54, in run
    package = self._setup_analysis_package()
  File "/tmpKoSNUL/analyzer.py", line 123, in _setup_analysis_package
    package_class = choose_package_class(self.config.file_type,
AttributeError: Config instance has no attribute 'file_type'
Traceback (most recent call last):
  File "/tmpKoSNUL/analyzer.py", line 192, in <module>
    success = analyzer.run()
  File "/tmpKoSNUL/analyzer.py", line 54, in run
    package = self._setup_analysis_package()
  File "/tmpKoSNUL/analyzer.py", line 123, in _setup_analysis_package
    package_class = choose_package_class(self.config.file_type,
AttributeError: Config instance has no attribute 'file_type'

In addition, I have pasted the Cuckoo log from the same analysis.

Cuckoo Sandbox 2.0.4
 www.cuckoosandbox.org
 Copyright (c) 2010-2017
 Checking for updates...
 You're good to go!
2017-09-17 19:54:18,573 [cuckoo.core.startup] INFO: Updated running task ID 5 status to failed_analysis
2017-09-17 19:54:18,594 [cuckoo.core.scheduler] INFO: Using "virtualbox" as machine manager
2017-09-17 19:54:20,458 [cuckoo.core.scheduler] INFO: Loaded 2 machine/s
2017-09-17 19:54:20,465 [cuckoo.core.scheduler] WARNING: As you've configured Cuckoo to execute parallel analyses, we recommend you to switch to a MySQL ora PostgreSQL database as SQLite might cause some issues.
2017-09-17 19:54:20,479 [cuckoo.core.scheduler] INFO: Waiting for analysis tasks.
2017-09-17 19:56:01,530 [cuckoo.core.scheduler] INFO: Starting analysis of URL "http://bing.com" (task #6, options "procmemdump=yes,route=none")
2017-09-17 19:56:01,558 [cuckoo.core.scheduler] INFO: Task #6: acquired machine cuckoo1 (label=cuckoo1)
2017-09-17 19:56:01,567 [cuckoo.auxiliary.mitm] INFO: Started mitm interception with PID 2991 (ip=192.168.56.1, port=50000).
2017-09-17 19:56:01,577 [cuckoo.auxiliary.sniffer] INFO: Started sniffer with PID 2992 (interface=vboxnet0, host=192.168.56.101)
2017-09-17 19:56:17,472 [cuckoo.core.guest] INFO: Starting analysis on guest (id=cuckoo1, ip=192.168.56.101)
2017-09-17 19:56:24,521 [cuckoo.core.guest] INFO: Guest is running Cuckoo Agent 0.7 (id=cuckoo1, ip=192.168.56.101)
2017-09-17 20:01:24,712 [cuckoo.core.guest] INFO: cuckoo1: end of analysis reached!
2017-09-17 20:01:26,032 [cuckoo.processing.behavior] WARNING: Analysis results folder does not contain any behavior log files.
2017-09-17 20:01:31,850 [cuckoo.core.scheduler] INFO: Task #6: reports generation completed
2017-09-17 20:01:31,870 [cuckoo.core.scheduler] INFO: Task #6: analysis procedure completed

Is this perhaps an issue with the agent.py not communicating with the host? I am able to ping between host and guest and Cuckoo does not throw an error when first run. Furthermore, I also achieve no results when analyzing a file. I am using the standard host-only network (vboxnet0) with the IP range of 192.168.56.0/24 on my VM. If you would like me to upload my config files, I'll be happy to do so.

Any info/assistance you may have is appreciated.

hollus commented 7 years ago

Apologies for the formatting of the logs, first time poster.

seantree commented 7 years ago

Hi @hollus,

Are you getting the same problem when submitting the mac malware sample to the sandbox?

Thanks & Regards Seantree

hollus commented 7 years ago

Hi @seantree thank you for replying. I appear to be receiving a new error now. See below the analysis.log:

2017-09-19 17:00:28,234 [root] DEBUG: Starting analyzer from /tmp_ApzEw 2017-09-19 17:00:28,234 [root] DEBUG: Storing results at: /tmp/tIqtuqzZh 2017-09-19 17:00:28,235 [root] ERROR: Traceback (most recent call last): File "/tmp_ApzEw/analyzer.py", line 192, in <module> success = analyzer.run() File "/tmp_ApzEw/analyzer.py", line 54, in run package = self._setup_analysis_package() File "/tmp_ApzEw/analyzer.py", line 124, in _setup_analysis_package self.config.file_name, **kwargs) File "/tmp_ApzEw/lib/core/packages.py", line 31, in choose_package_class "exist.".format(name)) Exception: Unable to import package "None": it does not exist. Traceback (most recent call last): File "/tmp_ApzEw/analyzer.py", line 192, in <module> success = analyzer.run() File "/tmp_ApzEw/analyzer.py", line 54, in run package = self._setup_analysis_package() File "/tmp_ApzEw/analyzer.py", line 124, in _setup_analysis_package self.config.file_name, **kwargs) File "/tmp_ApzEw/lib/core/packages.py", line 31, in choose_package_class "exist.".format(name)) Exception: Unable to import package "None": it does not exist.

I have also attached the cuckoo.log from the same analysis. cuckoo.log

Could it be that the mac vm cannot communicate with the host? I have another windows vm that works fine so I am not sure if this is the case.

seantree commented 7 years ago

Hi @hollus

Can you please share your conf folder in zip here. I will check it asap and can you tell me one more thing what commands did you used for port forwarding because ipfw is not working anymore in sierra OS if I am not wrong . Thanks & Regards Seantree

hollus commented 7 years ago

Hi @seantree I have uploaded the conf folder as requested. Please see if my configurations are correct. cuckoo conf.zip

Furthermore, I used pfctl for port forwarding. First I enabled forwarding using the command: sudo sysctl net.inet.ip.forwarding=1 net.inet.ip.forwarding: 1 -> 1

Then I created the rules I believed would successfully forward the traffic from the VM IP range (192.168.56.0/24) to the host box. Below is my /etc/pf.conf file:

`#

Default PF configuration file.

#

This file contains the main ruleset, which gets automatically loaded

at startup. PF will not be automatically enabled, however. Instead,

each component which utilizes PF is responsible for enabling and disabling

PF via -E and -X as documented in pfctl(8). That will ensure that PF

is disabled only when the last enable reference is released.

#

Care must be taken to ensure that the main ruleset does not get flushed,

as the nested anchors rely on the anchor point defined here. In addition,

to the anchors loaded by this file, some system services would dynamically

insert anchors into the main ruleset. These anchors will be added only when

the system service is used and would removed on termination of the service.

#

See pf.conf(5) for syntax.

#

#

com.apple anchor point

# scrub-anchor "com.apple/" nat-anchor "com.apple/" rdr-anchor "com.apple/" nat on {vboxnet0} proto {tcp, udp, icmp} from 192.168.56.0/24 to any -> {en1} nat on {vboxnet0} proto {tcp, udp, icmp} from 192.168.56.0/24 to any -> 192.168.56.0/24 pass from {lo0, 192.168.56.0/24} to any keep state dummynet-anchor "com.apple/" anchor "com.apple/*" load anchor "com.apple" from "/etc/pf.anchors/com.apple"`

Afterwards, I verified if no errors were given with these rules:

sudo pfctl -vnf /etc/pf.conf and here is my output:

`sudo pfctl -vnf /etc/pf.conf pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.

scrub-anchor "/" all fragment reassemble nat-anchor "/" all nat on vboxnet0 inet proto tcp from 192.168.56.0/24 to any -> 192.168.93.26 nat on vboxnet0 inet proto udp from 192.168.56.0/24 to any -> 192.168.93.26 nat on vboxnet0 inet proto icmp from 192.168.56.0/24 to any -> 192.168.93.26 nat on vboxnet0 inet proto tcp from 192.168.56.0/24 to any -> 192.168.56.0/24 nat on vboxnet0 inet proto udp from 192.168.56.0/24 to any -> 192.168.56.0/24 nat on vboxnet0 inet proto icmp from 192.168.56.0/24 to any -> 192.168.56.0/24 rdr-anchor "/" all pass inet6 from ::1 to any flags S/SA keep state pass on lo0 inet6 from fe80::1 to any flags S/SA keep state pass inet from 127.0.0.1 to any flags S/SA keep state pass inet from 192.168.56.0/24 to any flags S/SA keep state anchor "/" all dummynet-anchor "/*" all

Loading anchor com.apple from /etc/pf.anchors/com.apple anchor "/" all anchor "/" all`

Finally I enabled the rules using sudo pfctl -ef /etc/pf.conf

`sudo pfctl -ef /etc/pf.conf pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.

No ALTQ support in kernel ALTQ related functions disabled pfctl: pf already enabled`

Please let me know if you require more info or clarification. Thank you.

seantree commented 7 years ago

Hi @hollus,

I have created a sandbox like your environment so that I will also face the same problem and try to resolve it. So here are few more questions and 1 solution to cross check for you:

Questions:

  1. Did you analyse any mac sample successfully through web console?
  2. Your web console is able to detect the filetype of Mac sample and accept that sample like Windows sample does? or you have to forcefully submit the sample for mac in web console?
  3. Did you placed the mitm-ca-cert.p12 file in which location for tcpdump?

Solution: What I understood after seeing the config: check your virtualbox.conf file and I found there is snapshot name is missing and arrange the things in proper way:

like for windows: [windows] label ip snapshotname etc

[mac] label ip snapshotname etc and also try to change the name of your mac close you have given the .(dot) try to use _(underscore) may be it's creating some problem. . Thanks & Regards Seantree

hollus commented 7 years ago

Hi @seantree

  1. Did you analyse any mac sample successfully through web console?
  1. Your web console is able to detect the filetype of Mac sample and accept that sample like Windows sample does? or you have to forcefully submit the sample for mac in web console?
  1. Did you placed the mitm-ca-cert.p12 file in which location for tcpdump?

Lastly, I have done as you instructed and specified the snapshot name for each VM as well as removed the .(dot) and replaced with _(underscore)

seantree commented 6 years ago

Hi @hollus,

I have configured my mac machine according to your problem scenario but the 2 things which I am facing and not able to resolve I hope you will help me in that. So that I can move further with your problem. I have attached a zip file of mac malware it contains 3 mac malware and password is infected. Mac_Malware.zip

When I am submitting these malware's through the web console it's killing the VM and not taking the sample and not detecting while submitting in the web.

I don't know what happen suddenly. If you are able to analyze these malware's let me know. . Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

I was able to analyze all three malware samples through the web console. I have attached a screenshot of the analysis as well as all three reports. 1.zip 2.zip 3.zip screen shot 2017-10-09 at 1 32 22 pm

It appears that there were no errors in the analysis however the results appear to be mostly empty. Could you please verify if these were the expected results? Otherwise I could be mistaken.

Also, I think it is worth noting that Cuckoo identified all three samples as zlib compressed data in case this may help. Furthermore, I may have missed this before but during an analysis, the Mac VM agent.py reports a different xmlrpc error than what is listed in the analysis.log. I took another screenshot of this error. Please review and see if this may help.

xmlrpc error

Let me know if you need more information. Thank you.

seantree commented 6 years ago

HI @hollus ,

I am submitting my log error which I am facing ... while submitting the sample through web console following error I am facing:

2017-10-10 12:03:52,377 [cuckoo.machinery.virtualbox] DEBUG: Restoring virtual machine MACOS_1 to Snapshot1 2017-10-10 12:03:53,113 [cuckoo.core.scheduler] INFO: Loaded 1 machine/s 2017-10-10 12:03:53,130 [cuckoo.core.scheduler] INFO: Waiting for analysis tasks. 2017-10-10 12:06:23,645 [cuckoo.core.scheduler] DEBUG: Processing task #4 2017-10-10 12:06:23,655 [cuckoo.core.scheduler] INFO: Starting analysis of FILE "83059c756e4e79c537856108492f7512f5bf6bd51631e00960a40fb817140d9f" (task #4, options "route=none") 2017-10-10 12:06:23,684 [cuckoo.core.scheduler] INFO: Task #4: acquired machine MACOS_1 (label=MACOS_1) 2017-10-10 12:06:23,731 [cuckoo.auxiliary.sniffer] INFO: Started sniffer with PID 660 (interface=vboxnet0, host=192.168.56.101) 2017-10-10 12:06:23,732 [cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer 2017-10-10 12:06:23,786 [cuckoo.machinery.virtualbox] DEBUG: Starting vm MACOS_1 2017-10-10 12:06:24,000 [cuckoo.machinery.virtualbox] DEBUG: Restoring virtual machine MACOS_1 to Snapshot1 2017-10-10 12:06:34,690 [cuckoo.core.guest] INFO: Starting analysis on guest (id=MACOS_1, ip=192.168.56.101) 2017-10-10 12:06:35,849 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-10 12:06:36,857 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-10 12:06:37,875 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-10 12:06:38,879 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-10 12:06:39,910 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-10 12:06:40,919 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-10 12:06:41,977 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-10 12:06:42,038 [cuckoo.core.guest] INFO: Guest is running Cuckoo Agent 0.7 (id=MACOS_1, ip=192.168.56.101) 2017-10-10 12:06:42,068 [cuckoo.core.guest] DEBUG: Uploading analyzer to guest (id=MACOS_1, ip=192.168.56.101, monitor=latest, size=108827) 2017-10-10 12:06:42,101 [cuckoo.core.plugins] ERROR: Unable to stop auxiliary module: Sniffer Traceback (most recent call last): File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/plugins.py", line 162, in stop module.stop() File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/auxiliary/sniffer.py", line 154, in stop (out, err, faq("permission-denied-for-tcpdump")) CuckooOperationalError: Error running tcpdump to sniff the network traffic during the analysis; stdout = '' and stderr = "tcpdump: vboxnet0: You don't have permission to capture on that device\n((cannot open BPF device) /dev/bpf0: Permission denied)\n". Did you enable the extra capabilities to allow running tcpdump as non-root user and disable AppArmor properly (the latter only applies to Ubuntu-based distributions with AppArmor, see also https://cuckoo.sh/docs/faq/index.html#permission-denied-for-tcpdump)? 2017-10-10 12:06:42,160 [cuckoo.machinery.virtualbox] DEBUG: Stopping vm MACOS_1 2017-10-10 12:06:43,456 [cuckoo.core.scheduler] ERROR: Failure in AnalysisManager.run Traceback (most recent call last): File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 698, in run self.launch_analysis() File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 499, in launch_analysis self.guest_manage(options) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 394, in guest_manage self.guest_manager.start_analysis(options, monitor) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 465, in start_analysis self.upload_analyzer(monitor) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 372, in upload_analyzer self.determine_analyzer_path() File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 350, in determine_analyzer_path r = self.post("/mkdtemp", data={"dirpath": systemdrive}) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 311, in post r = session.post(url, *args, kwargs) File "/Users/ashutosh/venv/lib/python2.7/site-packages/requests/sessions.py", line 535, in post return self.request('POST', url, data=data, json=json, kwargs) File "/Users/ashutosh/venv/lib/python2.7/site-packages/requests/sessions.py", line 488, in request resp = self.send(prep, send_kwargs) File "/Users/ashutosh/venv/lib/python2.7/site-packages/requests/sessions.py", line 609, in send r = adapter.send(request, kwargs) File "/Users/ashutosh/venv/lib/python2.7/site-packages/requests/adapters.py", line 473, in send raise ConnectionError(err, request=request) ConnectionError: ('Connection aborted.', BadStatusLine("''",))

. Let me know if some clarification needed. . Thanks & Regards Seantree

seantree commented 6 years ago

After doing lots of research I found that it's Tcpdump problem it's need root permission to run the command so will you please tell me how you configured the tcpdump issue because in linux it's set by setcap and some few commands, I tried to add the /usr/sbin/tcpdump in sudoers list but it doesn't work for me still it requires sudo command to work. . So want to know what command shall I execute to fix this issue. . Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

I received the same error before and what worked for me was to change the tcpdump path specified in your auxiliary.conf. The default path for tcpdump in os x is /usr/local/sbin/tcpdump. I don't recall changing anything else as that seemed to have resolved that same error I was receiving.

Try that and let me know if it works.

hollus

seantree commented 6 years ago

Hi @hollus,

Still I am getting the same problem after fixing the path its generate the same problem it needs root permission as I mentioned previously if I run the only tcpdump in terminal it show the operation not permitted but when I use with sudo tcpdump it runs the tcpdump perfectly but not the cuckoo :( . I also tried to disabled everything in auxiliary.conf to check what else errors they are producing this time also it generates some error. Actually the thing is my cuckoo is not forwarding the sample to the guest machine what I think. If I am not wrong, I tried most of the things but none of them working but in my ubuntu and centos machine everything is working fine except in MAC OS X :( . This time seriously I need some helps to understand what's actually going on in my MACOS cuckoo.

Let me know if you have some solutions. . Thanks & Regards Seantree

seantree commented 6 years ago

I have also checked the connection between host and guest while doing pinging ... everything is fine.. both the machines are pinging each other perfectly.

SparkyNZL commented 6 years ago

make sure u disable app armor , and enable the ability to run tcpdump as none root ...

sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump

Then test it with ...

getcap /usr/sbin/tcpdump

You should get an output like...

/usr/sbin/tcpdump = cap_net_admin,cap_net_raw+eip

Sent from Pixel 2

On Wed, Oct 11, 2017 at 8:11 AM +1300, "hollus" notifications@github.com wrote:

Hi @seantree

I received the same error before and what worked for me was to change the tcpdump path specified in your auxiliary.conf. The default path for tcpdump in os x is /usr/local/sbin/tcpdump. I don't recall changing anything else as that seemed to have resolved that same error I was receiving.

Try that and let me know if it works.

hollus

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

seantree commented 6 years ago

Hi @SparkyNZL ,

In macos setcap is not installed and I tried to look into how to install setcap but didn't get any success if you have any commands how to install setcap in macosx sierra let me know I will try it. . Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

I apologize for my last response. There was something else I tried that may work for you. Since tcpdump is installed by default on OS X, it is difficult to get to to run as non root. What I did was I installed tcpdump via Homebrew. Once it was installed, I linked the OS X version of tcpdump to point to our Homebrew version. Since we installed tcpdump as a non root user via Homebrew, this eliminated the permissions issue.

  1. Install Homebrew via https://brew.sh/
  2. run the command brew install tcpdump
  3. point to the Homebrew version by running brew link --overwrite tcpdump
  4. run the final command brew unlink tcpdump && brew link tcpdump

You should see your permissions for tcpdump point to the Homebrew version as well as who is the owner of it. See screenshot below for reference

screen shot 2017-10-11 at 11 57 40 am

Please let me know if this worked for you or if you need further assistance.

seantree commented 6 years ago

Hi @hollus

Still I am unable to do that:

Warning: tcpdump 4.9.2 is already installed Ashutoshs-Mac-mini:~ ashutosh$ brew link --overwrite tcpdump Warning: Already linked: /usr/local/Cellar/tcpdump/4.9.2 To relink: brew unlink tcpdump && brew link tcpdump Ashutoshs-Mac-mini:~ ashutosh$ brew unlink tcpdump && brew link tcpdump Unlinking /usr/local/Cellar/tcpdump/4.9.2... 3 symlinks removed Linking /usr/local/Cellar/tcpdump/4.9.2... 3 symlinks created Ashutoshs-Mac-mini:~ ashutosh$ tcpdump tcpdump: ioctl(SIOCIFCREATE): Operation not permitted Ashutoshs-Mac-mini:~ ashutosh$ setcap -bash: setcap: command not found

Let me know if you have other solutions for this. . Thanks & Regards Seantree

seantree commented 6 years ago

Hi @hollus,

I have solved the TCPDUMP problem by disabling the SIP in Mac but now I am facing the another problem when I am submitting the sample through the web console its not throwing the sample to the guest machine & I have to submit the sample forcefully in web console it's not recognizing the sample by its type.

So below is the new error which I am facing now:

2017-10-12 17:02:49,943 [cuckoo.core.guest] INFO: Starting analysis on guest (id=MACOS_1, ip=192.168.56.101) 2017-10-12 17:02:50,951 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-12 17:02:51,959 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-12 17:02:52,967 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-12 17:02:53,975 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-12 17:02:54,980 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-12 17:02:55,987 [cuckoo.core.guest] DEBUG: MACOS_1: not ready yet 2017-10-12 17:02:57,030 [cuckoo.core.guest] INFO: Guest is running Cuckoo Agent 0.7 (id=MACOS_1, ip=192.168.56.101) 2017-10-12 17:02:57,040 [cuckoo.core.guest] DEBUG: Uploading analyzer to guest (id=MACOS_1, ip=192.168.56.101, monitor=latest, size=108827) 2017-10-12 17:02:57,082 [cuckoo.core.plugins] DEBUG: Stopped auxiliary module: MITM 2017-10-12 17:02:57,083 [cuckoo.core.plugins] DEBUG: Stopped auxiliary module: Sniffer 2017-10-12 17:02:57,083 [cuckoo.machinery.virtualbox] DEBUG: Stopping vm MACOS_1 2017-10-12 17:02:58,342 [cuckoo.core.scheduler] ERROR: Failure in AnalysisManager.run Traceback (most recent call last): File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 698, in run self.launch_analysis() File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 499, in launch_analysis self.guest_manage(options) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 394, in guest_manage self.guest_manager.start_analysis(options, monitor) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 465, in start_analysis self.upload_analyzer(monitor) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 372, in upload_analyzer self.determine_analyzer_path() File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 350, in determine_analyzer_path r = self.post("/mkdtemp", data={"dirpath": systemdrive}) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 311, in post r = session.post(url, *args, kwargs) File "/Users/ashutosh/venv/lib/python2.7/site-packages/requests/sessions.py", line 535, in post return self.request('POST', url, data=data, json=json, kwargs) File "/Users/ashutosh/venv/lib/python2.7/site-packages/requests/sessions.py", line 488, in request resp = self.send(prep, send_kwargs) File "/Users/ashutosh/venv/lib/python2.7/site-packages/requests/sessions.py", line 609, in send r = adapter.send(request, kwargs) File "/Users/ashutosh/venv/lib/python2.7/site-packages/requests/adapters.py", line 473, in send raise ConnectionError(err, request=request) ConnectionError: ('Connection aborted.', BadStatusLine("''",))


And Below is my pf.conf and I have executed the bootstrap_host.sh script for port forwarding and then I have made the changes in pf.conf according to your config, Have a look it and let me know if there is any correction is needed:

Default PF configuration file.

#

This file contains the main ruleset, which gets automatically loaded

at startup. PF will not be automatically enabled, however. Instead,

each component which utilizes PF is responsible for enabling and disabling

PF via -E and -X as documented in pfctl(8). That will ensure that PF

is disabled only when the last enable reference is released.

#

Care must be taken to ensure that the main ruleset does not get flushed,

as the nested anchors rely on the anchor point defined here. In addition,

to the anchors loaded by this file, some system services would dynamically

insert anchors into the main ruleset. These anchors will be added only when

the system service is used and would removed on termination of the service.

#

See pf.conf(5) for syntax.

#

#

#

com.apple anchor point

# scrub-anchor "com.apple/" nat-anchor "com.apple/" rdr-anchor "com.apple/" nat on {vboxnet0} proto {tcp, udp, icmp} from 192.168.56.0/24 to any -> {en0} nat on {vboxnet0} proto {tcp, udp, icmp} from 192.168.56.0/24 to any -> 192.168$ pass from {lo0, 192.168.56.0/24} to any keep state dummynet-anchor "com.apple/" anchor "com.apple/*" load anchor "com.apple" from "/etc/pf.anchors/com.apple"

Let me know if you have any solution for this. . Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

Everything seems correct to me. Perhaps double check your host interface? Mine is set to en1. Also what is the purpose of the dollar sign $ in the line nat on {vboxnet0} proto {tcp, udp, icmp} from 192.168.56.0/24 to any -> 192.168$? I have mine explicitly written on in my configuration.

Have you also tried connecting to the VM via curl? From your host, run the command:

curl 192.168.56.101:8000

That IP belongs to my Mac VM and the cuckoo agent is listening on port 8000. See my results below:

screen shot 2017-10-12 at 11 16 09 am

Can you please run the following commands and post their output for me to review?

sudo pfctl -vnf /etc/pf.conf

sudo pfctl -ef /etc/pf.conf

seantree commented 6 years ago

HI @hollus ,

Here is the output of the commands:

Ashutoshs-Mac-mini:etc ashutosh$ curl 192.168.56.101:8000 {"message": "Cuckoo Agent!", "version": "0.7", "features": ["execpy", "pinning", "logs", "largefile", "unicodepath"]}Ashutoshs-Mac-mini:etc ashutosh$ Ashutoshs-Mac-mini:etc ashutosh$ sudo pfctl -vnf /etc/pf.conf Password: pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.

scrub-anchor "/" all fragment reassemble nat-anchor "/" all nat on vboxnet0 inet proto tcp from 192.168.56.0/24 to any -> 192.168.1.98 nat on vboxnet0 inet proto udp from 192.168.56.0/24 to any -> 192.168.1.98 nat on vboxnet0 inet proto icmp from 192.168.56.0/24 to any -> 192.168.1.98 nat on vboxnet0 inet proto tcp from 192.168.56.0/24 to any -> 192.168.56.0/24 nat on vboxnet0 inet proto udp from 192.168.56.0/24 to any -> 192.168.56.0/24 nat on vboxnet0 inet proto icmp from 192.168.56.0/24 to any -> 192.168.56.0/24 rdr-anchor "/" all pass inet6 from ::1 to any flags S/SA keep state pass on lo0 inet6 from fe80::1 to any flags S/SA keep state pass inet from 127.0.0.1 to any flags S/SA keep state pass inet from 192.168.56.0/24 to any flags S/SA keep state anchor "/" all dummynet-anchor "/*" all

Loading anchor com.apple from /etc/pf.anchors/com.apple anchor "/" all anchor "/" all Ashutoshs-Mac-mini:etc ashutosh$ sudo pdfctl -ef /etc/pf.conf sudo: pdfctl: command not found Ashutoshs-Mac-mini:etc ashutosh$ sudo pfctl -ef /etc/pf.conf pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.

No ALTQ support in kernel ALTQ related functions disabled pf enabled Ashutoshs-Mac-mini:etc ashutosh$

seantree commented 6 years ago

HI @hollus,

Few more things just want to know did you used the following links to setup your MACOSX:

https://github.com/rodionovd/cuckoo-osx-analyzer

https://github.com/rodionovd/cuckoo-osx-analyzer/issues/6#issuecomment-101322097

Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

From the last command output, it looks like pf wasn't enabled yet otherwise it would have said pf already enabled.

Try running a sample again and let me know if you still receive an error.

Also, I did not use those links when setting up my OS X.

-hollus

seantree commented 6 years ago

Hi @hollus,

I have checked the sample running test again it's showing the same behavior which I have mentioned previously

2017-10-12 17:02:57,082 [cuckoo.core.plugins] DEBUG: Stopped auxiliary module: MITM 2017-10-12 17:02:57,083 [cuckoo.core.plugins] DEBUG: Stopped auxiliary module: Sniffer 2017-10-12 17:02:57,083 [cuckoo.machinery.virtualbox] DEBUG: Stopping vm MACOS_1 2017-10-12 17:02:58,342 [cuckoo.core.scheduler] ERROR: Failure in AnalysisManager.run Traceback (most recent call last): File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 698, in run self.launch_analysis() File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 499, in launch_analysis self.guest_manage(options) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/scheduler.py", line 394, in guest_manage self.guest_manager.start_analysis(options, monitor) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 465, in start_analysis self.upload_analyzer(monitor) File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 372, in upload_analyzer self.determine_analyzer_path() File "/Users/ashutosh/venv/lib/python2.7/site-packages/cuckoo/core/guest.py", line 350, in determine_analyzer_path

===================================================================

  1. I am able to ping both the machines host and guest successfully but the internet is not working on guest machine.

  2. Is there anything I have to execute on guest machine? Did you used dtrace on guest machine? I have only installed the python and pillow in the guest machine and ran the agent.py in terminal and took the running state of snapshot. I also ran the guest bootscript for setting network configuration properly, previously I have entered the Network settings manually which was the same.

Let me know if you have any solution . Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

Strangely enough, my guest machines are also unable to connect to the internet now. I checked the firewalls of my guest machines and they are disabled so this leads me to believe that the forwarding rules are not working. Perhaps they need to be changed and/or updated. I will work on this and update if I have an answer.

As for the guest machine itself, the only thing I have executed on the machine was the agent.py in the terminal. I also installed Pillow on them, disabled the firewalls and auto updates, and manually configured each VM's network configuration (static IP with DNS 8.8.8.8,8.8.4.4)

Apologies that this issue is taking so long to resolve

hollus commented 6 years ago

Hi @seantree

I ended up using the same forwarding rules from the link you posted earlier: https://github.com/rodionovd/cuckoo-osx-analyzer#on-the-host-side-os-x

First I flushed all my rules using sudo pfctl -F all

Then I disabled pf using sudo pfctl -df /etc/pf.conf

Next,I manually entered the forwarding rules in my /etc/pf.conf file: nat on en1 from vboxnet0:network to any -> (en1) pass inet proto icmp all pass in on vboxnet0 proto udp from any to any port domain keep state pass quick on en1 proto udp from any to any port domain keep state

Finally I checked the configuration for errors and enabled pf once more: sudo pfctl -vnf /etc/pf.conf sudo pfctl -ef /etc/pf.conf

My guest machines now have internet access. I'm afraid I don't know any solution as to why you are receiving that error in Cuckoo. Can I suggest flushing your rules and reenabling them once more? Also ensure the guest machine's firewall is disabled. Please let me know if you are able to connect your VMs to the internet.

jbremer commented 6 years ago

This is a rather long thread. Any takeaways or things that should be improved? Thanks.

seantree commented 6 years ago

Hi @hollus,

Sorry for the late reply, I have configured the PF.conf file according to your config but when I ran the sudo pfctl commands I received the following output let me know if everything is correct here or not

Ashutoshs-Mac-mini:etc ashutosh$ sudo pfctl -vnf /etc/pf.conf pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.

no IP address found for vboxnet0:network /etc/pf.conf:25: could not parse host specification Ashutoshs-Mac-mini:etc ashutosh$ sudo pfctl -ef /etc/pf.conf pfctl: Use of -f option, could result in flushing of rules present in the main ruleset added by the system at startup. See /etc/pf.conf for further details.

No ALTQ support in kernel ALTQ related functions disabled no IP address found for vboxnet0:network /etc/pf.conf:25: could not parse host specification pfctl: Syntax error in config file: pf rules not loaded

. Thanks & Regards SeanTree

seantree commented 6 years ago

And one more thing I have used en0 because my system has IP on en0 instead of en1

hollus commented 6 years ago

Hi @seantree

That output is not correct. There was no IP address found for your vboxnet0 interface. This is usually caused when no virtual machines are up and running. To fix this, simply power on the VM and the IP address will be assigned to your vboxnet0 interface. My IP address is set to 192.168.56.1.

Once you have an IP address assigned, re run the commands:

sudo pfctl -vnf /etc/pf.conf sudo pfctl -ef /etc/pf.conf Let me know your results. Thanks

-hollus

seantree commented 6 years ago

Hi @hollus

So after starting the VM here is the output of everything let me know what and where the issue is going.

  1. Check the below image after starting the VM I ran the pfctl -vnf and -ef commands: screen shot 2017-10-24 at 11 34 44 am

  2. Below is the screenshot of my host network info:

screen shot 2017-10-24 at 11 26 53 am

screen shot 2017-10-24 at 11 26 59 am

  1. Below is the screenshot of guest OS Network Info:

screen shot 2017-10-24 at 11 15 45 am

  1. Below is the screenshot of guest.py file in terminal in guest OS:

screen shot 2017-10-24 at 11 16 55 am

  1. Below is the screenshot of pinging guest machine to yahoo.com:

screen shot 2017-10-24 at 11 18 10 am

  1. Below is the screenshot of pinging 192.168.56.101 from host to guest:

screen shot 2017-10-24 at 11 27 29 am

  1. Below is the screenshot of pinging from guestOS to hostOS:

screen shot 2017-10-24 at 11 28 23 am

Now let me know if you have any solutions. . Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

Your pfctl command outputs look correct and is the same as mine. I should mention to double check if forwarding is enabled by running: sudo sysctl net.inet.ip.forwarding=1

If you see the output 0 -> 1 then that means forwarding wasn't enabled before which would explain why your guest VM can't ping host.

If forwarding was already enabled then something else is blocking traffic. Perhaps the firewall on the guest VM is turned on? If so, disable it. In addition, I have my guest VM DNS settings set to 8.8.8.8 and 8.8.4.4.

Also Remember to run the agent.py on the guest VM with admin privileges.

Please let me know if that helps, thank you

-hollus

seantree commented 6 years ago

Hi @hollus,

Thank you for reminding me for that command now I can ping from guest to host but still there is no internet running and I also update the DNS to 8.8.8.8 and 8.8.4.4.

Second thing Now when I am uploading the sample through web console still it's not detecting its filetype I am submitting the Mac sample forcefully, Now My MAC_Guest_VM able to upload the sample because I ran the agent.py through root permission but the thing is I am getting some error in agent.py when the sample is analysing it's need some dtrace packages below is the screenshot of that: Did you face the same problem?

screen shot 2017-10-25 at 4 27 31 pm

Below is the cuckoo log in zip file attachment and kindly check the screenshots: 12.zip

1

2

3

Thanks & Regards Seantree

seantree commented 6 years ago

Hi @hollus,

I have solved the some modules problem but now I am facing another problem on guest machine check the below screenshot is this problem need internet access?

screen shot 2017-10-27 at 3 40 03 pm

Let me know if you have some solution for this. . Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

Apologies for the late delay. The XMLRPC error issue you are having is the same one I am having and stuck with. I currently don't have a solution for this unfortunately. Any ideas you may have?

seantree commented 6 years ago

Hi @hollus

Thank you for the reply, I don't have any idea on this right now, but why are you getting this problem previously everything was working fine on your system if I am not wrong you were able to analyse the mac sample which I sent to you then suddenly why you are getting this error. Try to remember what major change you did on your system (If previously you were not getting this error). . Let me know if some idea you get to resolve this error otherwise this time we have to call @doomedraven and @jbremer to help us from this situation. . Thanks & Regards Seantree

seantree commented 6 years ago

Hi @hollus,

I was reading something on cuckoo I found this following article:

http://seclist.us/cuckoo-sandbox-v-1-2-released-is-an-automated-dynamic-malware-analysis-system.html

search for xmlrpc word in this article, it was describing regarding sharing folders. I didn't implement anything from this just sharing with you hope something would help us. . Thanks & Regards Seantree

hollus commented 6 years ago

Hi @seantree

I apologize for not being clearer. I always received this error when analyzing mac sample, however for some reason Cuckoo does not stop like you'd expect. Instead the analysis continues and eventually will finish. Please keep in mind that the analysis takes a long time and the finished results were mostly blank.

Also, I read the article you shared. I also did not implement any shared folders in my VMs.

doomedraven commented 6 years ago

shared folders is depricated, now agent do all that with uploading requried folders

seantree commented 6 years ago

Hi @doomedraven kindly help us, how to solve the above problem of xmlrpc we are getting this when we are analyzing the mac sample. . Thanks & Regards Seantree

doomedraven commented 6 years ago

no idea, i never had that problem and i don't have the mac os, but i forwarded that to persons who correctly setup the macos x vms

seantree commented 6 years ago

Thanks @doomedraven it would be great help for us if you will forward this issue asap to macos person because we are trying to solving this issue from the last 2 months as you can see this issue is posted on 18th september. . Waiting for some positive reply.

Thanks & Regards Seantree

doomedraven commented 6 years ago

i forwarded before replay to you :) but that is up to them if they will help or not, ican't help more here sorry

doomedraven commented 6 years ago

or maybe someone can share the vm? so i can setup it quickly and help

seantree commented 6 years ago

I can share the VM but that too big in size (here is a 10.8 version link https://drive.google.com/file/d/0BxBVjisqLRIrSTRySWJJUUlRZm8/view) but we used 10.12 sierra. It would be great if you can run the macos 10.12 Sierra guest and rest of the things are mentioned here that we did in our guest & host it will take max your 30-45 mins, If you want to give a try. . Thanks & Regards Seantree

doomedraven commented 6 years ago

space doesnt matter but having the vm i will can solve your problem :) at the moment i don't have so much time to create new vm, but is easier test existing vm, lets go by parts first

doomedraven commented 6 years ago

i will check it this night, but someone of you tested it with old agent? https://raw.githubusercontent.com/cuckoosandbox/cuckoo/legacy/agent/agent.py

seantree commented 6 years ago

No we didn't checked with the old agent

doomedraven commented 6 years ago

try it

seantree commented 6 years ago

HI @doomedraven,

I tried with the old version and getting this error: unable to import package none:

2017-10-27 15:15:18,954 [root] DEBUG: Starting analyzer from /Users/macos_1/egwlcylaf 2017-10-27 15:15:18,956 [root] DEBUG: Storing results at: /var/folders/vj/dh9mx4m503d6xn2r7lb_y0040000gn/T/ELXBtJGOtW 2017-10-27 15:15:18,959 [root] ERROR: Traceback (most recent call last): File "/Users/macos_1/egwlcylaf/analyzer.py", line 192, in success = analyzer.run() File "/Users/macos_1/egwlcylaf/analyzer.py", line 54, in run package = self._setup_analysis_package() File "/Users/macos_1/egwlcylaf/analyzer.py", line 124, in _setup_analysis_package self.config.file_name, kwargs) File "/Users/macos_1/egwlcylaf/lib/core/packages.py", line 31, in choose_package_class "exist.".format(name)) Exception: Unable to import package "None": it does not exist. Traceback (most recent call last): File "/Users/macos_1/egwlcylaf/analyzer.py", line 192, in success = analyzer.run() File "/Users/macos_1/egwlcylaf/analyzer.py", line 54, in run package = self._setup_analysis_package() File "/Users/macos_1/egwlcylaf/analyzer.py", line 124, in _setup_analysis_package self.config.file_name, kwargs) File "/Users/macos_1/egwlcylaf/lib/core/packages.py", line 31, in choose_package_class "exist.".format(name)) Exception: Unable to import package "None": it does not exist.