Android VM got killed directly after Start

mr-gosh commented 8 years ago

Hey, our Setup based on the git Repo starts the AVD. We used the CuckooDroid-Docs for Preparing the Guest (Android Emulator). But the VM shuts down after a few seconds with the following output. What could cause this?

2015-12-08 09:10:24,898 [lib.cuckoo.core.scheduler] INFO: Using "avd" as machine manager
2015-12-08 09:10:25,007 [lib.cuckoo.core.scheduler] INFO: Loaded 1 machine/s
2015-12-08 09:10:25,030 [lib.cuckoo.core.scheduler] INFO: Unable to determine the network interface for VM with name aosx_1, Cuckoo will not be able to give it full internet access or route it through a VPN! Please define a default network interface for the machinery or define a network interface for each VM.
2015-12-08 09:10:25,030 [lib.cuckoo.core.scheduler] INFO: Waiting for analysis tasks.
2015-12-08 09:11:57,382 [lib.cuckoo.core.scheduler] INFO: Starting analysis of FILE "Amazon_App.apk" (task #2, options "route=none")
2015-12-08 09:11:58,936 [lib.cuckoo.core.scheduler] INFO: File already exists at "/home/ra/d/cuckoo/storage/binaries/f6e9cf15c124c37d66982cde050a0330e6898e7ddc85f5949ff86b0973108730"
2015-12-08 09:11:58,996 [lib.cuckoo.core.scheduler] INFO: Task #2: acquired machine aosx_1 (label=aosx_1)
2015-12-08 09:11:59,016 [lib.cuckoo.common.objects] WARNING: Error extracting package and main activity: unpack requires a string argument of length 4.
2015-12-08 09:13:08,551 [lib.cuckoo.core.guest] INFO: Starting analysis on guest (id=aosx_1, ip=127.0.0.1)
2015-12-08 09:13:08,746 [modules.machinery.avd] INFO: Stopping AVD listening on port 5554
2015-12-08 09:13:09,842 [lib.cuckoo.core.scheduler] ERROR: Failure in AnalysisManager.run
Traceback (most recent call last):
  File "/home/ra/d/cuckoo/lib/cuckoo/core/scheduler.py", line 439, in run
    success = self.launch_analysis()
  File "/home/ra/d/cuckoo/lib/cuckoo/core/scheduler.py", line 326, in launch_analysis
    self.guest_manage(options)
  File "/home/ra/d/cuckoo/lib/cuckoo/core/scheduler.py", line 254, in guest_manage
    guest.start_analysis(options, monitor)
  File "/home/ra/d/cuckoo/lib/cuckoo/core/guest.py", line 390, in start_analysis
    r.json().get("version"), self.vmid, self.ipaddr)
  File "/usr/local/lib/python2.7/dist-packages/requests/models.py", line 819, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 488, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 389, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
JSONDecodeError: Expecting value: line 2 column 1 (char 1)

Thanks for the Hints and this is possibly a duplicate of #696

jbremer commented 8 years ago

Ah interesting. This bug is unrelated to #696 for the reasons outlined by my last comment on that issue :-) The issue here seems to be that we've recently been working on a new, improved, Cuckoo Agent, and while the Agent included with the Android analyzer is based on the old Agent, the detection for this Agent doesn't seem to work.. if you can follow that train of thought. (So Windows analysis supports old & new Agent, Android analysis supports old, but for some reason the old Agent is not detected properly for the Android analyzer).

Could you start the Android emulator, make sure it runs the Agent, and then curl guest_ip:8000 (so probably under Android that should be curl 10.0.2.2:8000). Please let me know the content and http code that returns :-)

mr-gosh commented 8 years ago

OK - we poked around a bit:

Tested with agent.sh file from cuckoo-droid agent.py file from idanr1986/cuckoo-droid and cuckoobox/cuckoo

activated the VM by hand (so it doesn't shut down with the above mentioned error) and did an

adb shell /data/local/agent.sh

than we mapped VM port 8000 to Host port 5001 an curled a bit:

$ curl localhost:5001

<head>
<title>Error response</title>
</head>
<body>
<h1>Error response</h1>
<p>Error code 501.
<p>Message: Unsupported method ('GET').
<p>Error code explanation: 501 = Server does not support this operation.
</body>

benjo9 commented 8 years ago

We now tried agent.py from jbremer/agent. That looks better from curl response, but error message from cuckoo tests stays the same. $ curl localhost:5001 {"message": "Cuckoo Agent!", "version": "0.1"}

shell with agent.py 10.0.2.2 - - [09/Dec/2015 13:32:00] "GET / HTTP/1.1" 200 -

jbremer commented 8 years ago

Not so fast, I didn't say that the Android Analyzer supports the new Agent ;-) In fact, the curl localhost:5001 is correct - XMLRPC is expected to return 501 when requesting /. That's also what the Cuckoo magic is based on to decide whether to take the old or the new Agent, see also https://github.com/cuckoobox/cuckoo/blob/master/lib/cuckoo/core/guest.py#L379-L380. Having said that it seems weird to me that you were getting the new Agent selection in the first place - did you correctly setup all the IP addresses as advertised in the documentation?

threedanes commented 8 years ago

Quick question, does your host have the http_proxy set?
As for my issue, thank you jbremer btw, #696 I removed the environment http_proxy and low and behold it started working.

I should note that I am currently waiting for a dedicated "dirty line". While I am waiting for that to be installed I am using my lab network which obviously goes through a proxy. Just trying to get this up, documented and play with it.

benjo9 commented 8 years ago

Ok, i did not expect 501 to be correct ;) IP settings are as in documentation.

cuckoo.conf ip = 127.0.0.1

avd.conf

emulator_port = 5554
resultserver_ip = 10.0.2.2
resultserver_port = 2042

No Proxy or http_proxy present. Host Ip Adresse is 192.168.1.x If i need to connect directly to a Service on Android VM i can use adb forward or Telnet to Emulator and redir. Maybe we should start with a older Version first and not with latest repo?

jbremer commented 8 years ago

You could try an older version, i.e., our 1.2 release on top of which you can directly apply the patches from Cuckoo Droid as described in their repository. If that works then something is going wrong with the latest updates regarding the old/new Agent. Otherwise there's some more debugging to do on your side, I suppose.

https://github.com/idanr1986/cuckoo-droid#cuckoodroid---automated-android-malware-analysis

benjo9 commented 8 years ago

We got 1.2 running and analysis working. Don't know yet why new version is not working. Trying to find out what happens or not, but maybe wait for changes with new Agent. And i am not familiar with Python, unfortunately.

botherder commented 8 years ago

@jbremer Any updates on this?

jbremer commented 8 years ago

No, will have to check back later.

mr-gosh commented 8 years ago

hey @jbremer - we really wanna implement this into our pipleline - is there anything going to happen? Or do you have an other idea?

snemes commented 8 years ago

@mr-gosh, I think some of the recent changes fixed this issue. I was able to set up Android analysis. There are still some missing features (e.g. on the web UI), but it definitely should work now. See https://github.com/cuckoosandbox/cuckoo/issues/926 for details.

mr-gosh commented 8 years ago

@benjo9 did you re-check this in our setup?

blacvortex commented 7 years ago

I get $ curl localhost:5001 {"message": "Cuckoo Agent!", "version": "0.1"} as well but the analysis does not complete for me. I see the "analysis still running" prompt and then a timeout. I feel the analyzer part is not working for me as clearly the cuckoo agent is running fine.

Any pointers ?

lovina37 commented 7 years ago

@jbremer @botherder

cuckoosandbox / cuckoo

Android VM got killed directly after Start #697