search

programa-stic / Marvin-dynamic-Analyzer

Dynamic android vulnerability scanner using OpenNebula and Android-x86 emulators.
42 stars 23 forks source link

from libmproxy.protocol.http import decoded error #1

Open cnscyy opened 7 years ago

cnscyy commented 7 years ago

I have install mitmproxy using apt-get mitmproxy, the version is 0.15.1. it don't work when i install mitmproxy using pip install. Then i run python VMclient.py Traceback ImportError: cannot import name decoded. The linux OS is Ubunt16.4

This is an error in mitmproxy, i want ask what envirment do you use? do you meet the same question?

jheguia commented 7 years ago

Hi, We're using mitmproxy version 0.11.3. They've been changing the API, so it's very likely that newer versions will not work with the dynamic analyzer. I'll try and change it to the new version, but I can't give you time estimates for that.

Cheers, Juan

2016-12-01 5:18 GMT-03:00 cnscyy notifications@github.com:

I have install mitmproxy using apt-get mitmproxy, the version is 0.15.1. it don't work when i install mitmproxy using pip install. Then i run python VMclient.py Traceback ImportError: cannot import name decoded. The linux OS is Ubunt16.4

This is an error in mitmproxy, i want ask what envirment do you use? do you meet the same question?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qiGGhWuXH_t5dMh0Ap1kaX3hJelmks5rDoK_gaJpZM4LBHMD .

cnscyy commented 7 years ago

Thanks for answer my question. I use the same version of mitmproxy as yours, it did not appear this question. Than you a lot.

cnscyy commented 7 years ago

hello，in the dynamic analyzer, can i use android X86 4.4 or higher version？

jheguia commented 7 years ago

There are two problems with that: Cydia Substrate support and fixed network support. I was able to make a version of Toqueton that uses Xposed Framework instead of Cydia Substrate, but the newer images insisted on raising the dhcp daemon and changing the routes, thus evading the proxy. I haven't found a way to get the network interface to use fixed address and routes.

Best regards, Juan

2016-12-16 2:30 GMT-03:00 cnscyy notifications@github.com:

hello，in the dynamic analyzer, can i use android X86 4.4 or higher version？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-267520276, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qjobZXLl4wQpfPiOrglKSy4PNfMKks5rIiHZgaJpZM4LBHMD .

cnscyy commented 7 years ago

Thank you for your patience！ I have another question about this project. the VM_MANAGER_IP、REPORTER_IP、DOWNLOAD_APK_SITE and frontpage ip are the same or different? if they are the same, is just the port number different?

cnscyy commented 7 years ago

this project--Marvin-dynamic-Analyzer can run in the windows? it can be achieved with serveral vms instead of OpenNebula?

jheguia commented 7 years ago

Hi, VM_MANAGER_IP is the address where the server is running. REPORTER_IP is where the client is running, and DOWNLOAD_APK_SITE is where the Marvin front-end is running. You can run the different parts of Marvin in different machines, or in the same machine.

Cheers, Juan

2016-12-22 6:25 GMT-03:00 cnscyy notifications@github.com:

Thank you for your patience！ I have another question about this project. the VM_MANAGER_IP、REPORTER_IP、DOWNLOAD_APK_SITE and frontpage ip are the same or different? if they are the same, is just the port number different?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-268756194, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qmis3EhXJW_VzjHxuHvPRfWWIl9Hks5rKkIdgaJpZM4LBHMD .

jheguia commented 7 years ago

It will not work "out of the box": it will require minor modifications to the startup scripts, figuring out how to redirect traffic to the proxy, and maybe do some changes to the calls to "sys" and the threads part. I think it's better if you run it in a Linux VM. As for replacing OpenNebula, any VM solution that lets you restore "live" snapshots programmatically should work fine.

Best regards, Juan

2016-12-22 7:20 GMT-03:00 cnscyy notifications@github.com:

this project--Marvin-dynamic-Analyzer can run in the windows? it can be achieved with serveral vms instead of OpenNebula?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-268768581, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qtemgBFNvS4TEIAIxGOJFj-fa5jtks5rKk7xgaJpZM4LBHMD .

cnscyy commented 7 years ago

Hello，thanks a lot. I will run it in Linux VMs. when i use emulator_setup.sh to root the android-x86 4.3, it always prompts "Permission denied":

mount: Permission denied

mkdir failed for /system/bin/.ext, Read-only file system

Unable to chmod /system/xbin/su: Read-only file system

remount failed: Permission denied

what should i do about this?

./emulator_setup.sh SSL 192.168.137.102 192.168.137.1 Run with {SSL/NOSSL} {EMULATOR-IP} {GATEWAY-IP-FOR-EMULATOR} as args SSL 192.168.137.102 192.168.137.1 Using emulator address 192.168.137.102:5556

connected to 192.168.137.102:5556 192.168.137.102:5556 adb connect 192.168.137.102:5556 adbd is already running as root adb -s 192.168.137.102:5556 root already connected to 192.168.137.102:5556 adb connect 192.168.137.102:5556 push: emulator_support_files/Android-x86-RootScript-4.3/system/etc/init.sh -> /data/local/system/etc/init.sh push: emulator_support_files/Android-x86-RootScript-4.3/system/bin/.ext/.su -> /data/local/system/bin/.ext/.su push: emulator_support_files/Android-x86-RootScript-4.3/system/bin/README -> /data/local/system/bin/README push: emulator_support_files/Android-x86-RootScript-4.3/system/app/Superuser.apk -> /data/local/system/app/Superuser.apk push: emulator_support_files/Android-x86-RootScript-4.3/system/xbin/su -> /data/local/system/xbin/su push: emulator_support_files/Android-x86-RootScript-4.3/system/xbin/daemonsu -> /data/local/system/xbin/daemonsu push: emulator_support_files/Android-x86-RootScript-4.3/install-device.sh -> /data/local/install-device.sh 7 files pushed. 0 files skipped. 3413 KB/s (2628119 bytes in 0.751s)

Root script for Android 4.3 By Quinny899 @ XDA Root by Chainfire @ XDA

Script loaded Installing to device from device Mounting... mount: Permission denied Removing old files... rm failed for /system/xbin/daemonsu, Read-only file system rm failed for /system/xbin/su, Read-only file system rm failed for /system/app/Superuser.apk, Read-only file system Copying files... mkdir failed for /system/bin/.ext, Read-only file system cp: /system/bin/.ext is not a directory cp: /system/xbin/su: Read-only file system cp: /system/xbin/daemonsu: Read-only file system cp: /system/app/Superuser.apk: Read-only file system install-device.sh[25]: can't create /system/etc/init.sh: Read-only file system Setting permissions... Unable to chmod /system/xbin/su: Read-only file system Unable to chmod /system/xbin/daemonsu: No such file or directory Unable to chmod /system/bin/.ext/.su: No such file or directory Unable to chmod /system/bin/.ext: No such file or directory Unable to chmod /system/etc/init.sh: Read-only file system Cleaning up... Finished. You need to reboot for root to be available remount failed: Permission denied sed: can't create temp file '/system/etc/init.shac2367': Read-only file system /system/bin/sh: can't create /system/etc/init.sh: Read-only file system /system/bin/sh: can't create /system/etc/init.sh: Read-only file system /system/bin/sh: can't create /system/etc/init.sh: Read-only file system /system/bin/sh: can't create /system/etc/init.sh: Read-only file system /bin/.ext is not a directory cp: /system/xbin/su: Read-only file system cp: /system/xbin/daemonsu: Read-only file system cp: /system/app/Superuser.apk: Read-only file system install-device.sh[25]: can't create /system/etc/init.sh: Read-only file system Setting permissions...

cnscyy commented 7 years ago

the android x86 system is android-x86-4.3-20130725.iso

cnscyy commented 7 years ago

Hello，sorry for bother you again! the question of rooting android x86 4.3 has been solved. now, i still have some question to ask you. is the front-end combine the static-analyzer with dynamic-analyzer? i have not find link for dynamic-analyzer in the front pages.now the static-analyzer and dynamic-analyzer use the same database, but work independently at the same ip but different ports.

jheguia commented 7 years ago

The front end, the static analyzer and the dynamic analyzer all use the same DB backend, so you can see the results from the front end. The dynamic analyzer runs as stand alone; in fact you can have several clients for a server (as well as several Android VMs). The static analyzer is run from the front end, but is quite self-contained.

Cheers, Juan

2017-01-10 23:44 GMT-03:00 cnscyy notifications@github.com:

Hello，sorry for bother you again! the question of rooting android x86 4.3 has been solved. now, i still have some question to ask you. is the front-end combine the static-analyzer with dynamic-analyzer? i have not find link for dynamic-analyzer in the front pages.now the static-analyzer and dynamic-analyzer use the same database, but work independently at the same ip but different ports.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-271762795, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qifLBQIgPuPJMh__S45VOv_z0v6uks5rREIFgaJpZM4LBHMD .

cnscyy commented 7 years ago

hello，again. now, i have run the VMmanage.py and VMclient.py. the REPORTER_IP='localhost:8081', i haven't see any code about the REPORTER_IP, when VMclient.py is running, it cann't send report to the ip, is there something i ignore? when VMclient.py running, usingTrigger.py and Analyzer.py to analyze the apk, i cann't see anything in the results, the report: {'trigger': {}, 'analyzer': {}} is still empty. i want to ask it will feedback the results when testing in the android system. and is there something more i shoud do? thanks for your patient!

jheguia commented 7 years ago

Hi, REPORTER_IP is used by Utils.get_reporter(), which itself is called by functions in Trigger.py and Analyzer.py. Perhaps you don't have forwarding enabled, or the iptables set? Check client/client_setup.sh and check that: a) You ran it as root b) The network interface mentioned (eth0) is the one your machine uses and there are no more network interfaces. c) The Android emulator routing tables have as default gateway the machine where you're running the client.

You can check if everything works by running mitmproxy -T and accessing some unencrypted page from the Android browser: if it's working, the traffic should show up in mitmproxy.

Cheers, Juan

2017-01-19 7:18 GMT-03:00 cnscyy notifications@github.com:

hello，again. now, i have run the VMmanage.py and VMclient.py. the REPORTER_IP='localhost:8081', i haven't see any code about the REPORTER_IP, when VMclient.py is running, it cann't send report to the ip, is there something i ignore? when VMclient.py running, usingTrigger.py and Analyzer.py to analyze the apk, i cann't see anything in the results, the report: {'trigger': {}, 'analyzer': {}} is still empty. i want to ask it will feedback the results when testing in the android system. and is there something more i shoud do? thanks for your patient!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-273734685, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qvgDaD3w88G0RwFZaVN224iOKuoLks5rTzhygaJpZM4LBHMD .

cnscyy commented 7 years ago

hi, thanks again! through Analyzer.py call the StorageAnalyzer(self).analyze_storage(), in StorageAnalyzer.py for keyword, value in self.sensitiveValues().iteritems(): if value in content: self.analyzer.add_to_report(self.get_filter_id(), "Found plaintext {0} (value {1}) in file {2}".format(keyword, value, file)) the self.sensitiveValues().iteritems() is the FUZZER_VALUES in the settings.py, FUZZER_VALUES={ 'PASSWORD': 's3cr3tpass', 'MAIL' : 'fakeemailandroid@gmail.com', 'PHONE' : '1112341234', 'CONTACTNAME' : 'C0ntactFuzz', 'CONTACTPHONE' : '1107060504' } for every time to compare the value with content, it is always different, so it never add anything to the report. does the FUZZER_VALUES should be configed in some rules? and in the project, i only see the Analyzer.py call the StorageAnalyzer, don't find the call to other analyzer file? is the something i don't do correctly?

cnscyy commented 7 years ago

hi, thanks again! through Analyzer.py call the StorageAnalyzer(self).analyze_storage(), in StorageAnalyzer.py for keyword, value in self.sensitiveValues().iteritems(): if value in content: self.analyzer.add_to_report(self.get_filter_id(), "Found plaintext {0} (value {1}) in file {2}".format(keyword, value, file)) the self.sensitiveValues().iteritems() is the FUZZER_VALUES in the settings.py, FUZZER_VALUES={ 'PASSWORD': 's3cr3tpass', 'MAIL' : 'fakeemailandroid@gmail.com', 'PHONE' : '1112341234', 'CONTACTNAME' : 'C0ntactFuzz', 'CONTACTPHONE' : '1107060504' } for every time to compare the value with content, it is always different, so it never add anything to the report. does the FUZZER_VALUES should be configed in some rules? and in the project, i only see the Analyzer.py call the StorageAnalyzer, don't find the call to other analyzer file? is the something i don't do correctly?

jheguia commented 7 years ago

Hi, The idea is that Marvin-Toqueton inputs those values when an app asks for a password or email, or even reads data from the phone such as the phone number. If those values are found in the storage files, we flag the app as having insecure storage. Maybe the app you're analyzing doesn't do that. What you should probably see if the Fuzzing Helper is installed, is these values being used as input to the app in the emulator, and the URLs it accesses in the shell where you run the client.

Cheers, Juan

2017-01-20 7:52 GMT-03:00 cnscyy notifications@github.com:

hi, thanks again! through Analyzer.py call the StorageAnalyzer(self).analyze_storage(), in StorageAnalyzer.py for keyword, value in self.sensitiveValues().iteritems(): if value in content: self.analyzer.add_to_report(self.get_filter_id(), "Found plaintext {0} (value {1}) in file {2}".format(keyword, value, file)) the self.sensitiveValues().iteritems() is the FUZZER_VALUES in the settings.py, FUZZER_VALUES={ 'PASSWORD': 's3cr3tpass', 'MAIL' : 'fakeemailandroid@gmail.com', 'PHONE' : '1112341234', 'CONTACTNAME' : 'C0ntactFuzz', 'CONTACTPHONE' : '1107060504' } for every time to compare the value with content, it is always different, so it never add anything to the report. does the FUZZER_VALUES should be configed in some rules? and in the project, i only see the Analyzer.py call the StorageAnalyzer, don't find the call to other analyzer file? is the something i don't do correctly?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-274044017, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qmnVC0cDrT0ns3rD45wqScAGv-61ks5rUJHwgaJpZM4LBHMD .

cnscyy commented 7 years ago

hello, the Fuzzing Helper is installed well. maybe the values is not used as input to the app in emulator. using the values as input to the app is automatic or manual？？？ when i run ./client. sh, it is in a loop to do python VMClient.py. this will analyze the vulnerabilities again and again, even to the same vulnerability, is this all right？？？

jheguia commented 7 years ago

Hi, Have you installed Cydia Substrate? The FuzzingHelper needs it to work properly. The values are in the file assets/privacy.json. client.sh will ask the server for a vulnerability to check, and the server will choose one at random, so it is perfectly possible that it checks the same vuln repeatedly (especially if there is only one). Since the search has a random component, it might be that the fuzzer needs to check an app several times before it finds a way to exploit the vuln. There is a limit to the number of times it will do the test: it's hardcoded in client.py.

Cheers, Juan

2017-01-22 23:04 GMT-03:00 cnscyy notifications@github.com:

hello, the Fuzzing Helper is installed well. maybe the values is not used as input to the app in emulator. using the values as input to the app is automatic or manual？？？ when i run ./client. sh, it is in a loop to do python VMClient.py. this will analyze the vulnerabilities again and again, even to the same vulnerability, is this all right？？？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-274380703, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qks1AocpG31SZVvQnTUjPNh3hOYlks5rVArGgaJpZM4LBHMD .

cnscyy commented 7 years ago

hi, thanks again. i have installed Cydia Substrate. image In assets/privacy.json, the content is the same as FUZZER_PRIVACY_VALUES. but there is no FUZZER_VALUES in assets/privacy.json. In StorageAnalyzer.py, it compares the FUZZER_VALUES, should i add the FUZZER_VALUES to assets/privacy.json ??? In static analyzer, there are vulnerabilities SSL_CUSTOM_TRUSTMANAGER. when tested in dynamic analyzer, there is no vulnerability. didn't call handle_request() in SSLAnalyzer.py. is there something wrong?

jheguia commented 7 years ago

Hi, The values in FUZZER_VALUES don't show up in privacy.json because they are hardcoded (sorry). As for the dynamic analysis, sometimes the custom trust manager is only for debug mode or works well anyway (such as when you do certificate pinning). Dynamic analysis is there precisely because static analysis gives too many false positives. That said, handle_request() not being called could mean that either the certificates are being properly validated or that the forwarding isn't working: did you see mention of any traffic when running the client?

Cheers, Juan

2017-02-07 0:35 GMT-03:00 cnscyy notifications@github.com:

hi, thanks again. i have installed Cydia Substrate. [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170206180608.png In assets/privacy.json, the content is the same as FUZZER_PRIVACY_VALUES. but there is no FUZZER_VALUES in assets/privacy.json. In StorageAnalyzer.py, it compares the FUZZER_VALUES, should i add the FUZZER_VALUES to assets/privacy.json ??? In static analyzer, there are vulnerabilities SSL_CUSTOM_TRUSTMANAGER. when tested in dynamic analyzer, there is no vulnerability. didn't call handle_request() in SSLAnalyzer.py. is there something wrong?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-277892198, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qr-RQvinw0zgHMKu3T090dmkF6XEks5rZ-aNgaJpZM4LBHMD .

cnscyy commented 7 years ago

hi, thanks again. i run the client and server in the same VM. when i run ./client.sh, i can see the traffic in VMmanage and VMclient, such as: image image but when i run ./client.sh and VMmanage.py, it can't run mitmproxy -T properly. image is there someting wrong in forwarding?

jheguia commented 7 years ago

Well, you can't run mitmproxy when the client is running because the client is already running mitmproxy, so that's not the problem. Thing is, the emulator's traffic has to be routed via the machine where client.sh runs, and that machine has to have forwarding enabled and the 80 and 443 ports forwarded to port 8080. When you run client.sh, after "connected to adb as root", the messages on screen should read:

Installing app App installed Setting gateway Gateway set Running FuzzingTrigger

and then there should be a list of URIs visited, maybe with some lines starting with "request path".

Then some messages about POST requests and "already closed trigger", "already closed analyzer", "already closed reporter".

Do you see all of that? If there isn't a list of URIs, you can try not running the client and checking that mitmproxy captures regular HTTP traffic. Do make sure the time on the emulator is correctly set or the browser will drive you crazy with messages about invalid certificates.

Cheers, Juan

2017-02-08 3:06 GMT-03:00 cnscyy notifications@github.com:

hi, thanks again. i run the client and server in the same VM. when i run ./client.sh, i can see the traffic in VMmanage and VMclient, such as: [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170208111334.png [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170208112736.png but when i run ./client.sh and VMmanage.py, it can't run mitmproxy -T properly. [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170208135113.png is there someting wrong in forwarding?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-278237813, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qulPcV49NtN1ygRQ8a-LQKZxbuBTks5raVtNgaJpZM4LBHMD .

cnscyy commented 7 years ago

when i run ./client.sh, the message on the screen is: image the trigger and analyzer are empty. i found that it didn't call the handle_request in SSLAnalyzer.py is there some question?

jheguia commented 7 years ago

It does seem that traffic isn't reaching the analyzer. Run mitmproxy -T (the client should not be running), start a browser in the emulator, and try to access some HTTP page. mitmproxy should show you the requests and responses, at least for the nonencrypted accesses. If it doesn't, it could be several things:

If this is not the case, perhaps you didn't run client_setup.sh as root before starting the client for the first time.

Tell me what results you get.

Cheers, Juan

2017-02-09 3:16 GMT-03:00 cnscyy notifications@github.com:

when i run ./client.sh, the message on the screen is: [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170209135823.png the trigger and analyzer are empty. i found that it didn't call the handle_request in SSLAnalyzer.py is there some question?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-278557079, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qmwCtVvvWffoHRSkRBJxuZRDS9qGks5raq8_gaJpZM4LBHMD .

cnscyy commented 7 years ago

Hello， i have run client_setup.sh as root. image before i run client.sh, the route about emulator is: image when i run client.sh, it will excute set_localhost_as_gateway, the route about emulator is: image the ip of the client is:192.168.137.111 when i run mitmproxy -T and try to access some HTTP page, the result is: image

i also find that: when excute the cmd of monkey, the emulator will always do the monkey test and never stopped. the result of the dynamic analyze is still empty. thanks again!

jheguia commented 7 years ago

Try running the application with mitmproxy running. If nothing shows up, it could be that the app communicates exclusively via https, and does proper certificate validation. In that case, no traffic should show up.

Cheers, Juan

2017-02-10 6:52 GMT-03:00 cnscyy notifications@github.com:

Hello， i have run client_setup.sh as root. [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170210095035.png before i run client.sh, the route about emulator is: [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170210172235.png when i run client.sh, it will excute set_localhost_as_gateway, the route about emulator is: [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170210173309.png the ip of the client is:192.168.137.111 when i run mitmproxy -T and try to access some HTTP page, the result is: [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170210155400.png

i also find that: when excute the cmd of monkey, the emulator will always do the monkey test and never stopped. the result of the dynamic analyze is still empty. thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-278902105, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qiRgPPsW9axFQestGRlThQdO0pflks5rbDNwgaJpZM4LBHMD .

cnscyy commented 7 years ago

when the app was tested by monkey test, there is some http information in mitmproxy. image but it is still empty after run dynamic analyze, does this means the vuln analyzed by static analyzer is not realy existed? thanks again!

jheguia commented 7 years ago

What it means is that the dynamic analyzer could not confirm the vuln. It could still be there. Although all the traffic I see is HTTP: you can't verify correct certificate treatment from plaintext traffic. Check if you have a file called "[package_name]_network_traffic" and whether it's empty or not.

Cheers, Juan

2017-02-13 7:02 GMT-03:00 cnscyy notifications@github.com:

when the app was tested by monkey test, there is some http information in mitmproxy. [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170213152727.png but it is still empty after run dynamic analyze, does this means the vuln analyzed by static analyzer is not realy existed? thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-279342345, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qjd4XDREaJTJn5f_xpa-52PUz4B0ks5rcCosgaJpZM4LBHMD .

jheguia commented 7 years ago

It only means the analyzer was not able to trigger the vuln; it could still be there. Tomorrow I can send you some apps with vulns that should get triggered easily.

Cheers, Juan

2017-02-13 22:16 GMT-03:00 cnscyy notifications@github.com:

when the app was tested by monkey test, there is some http information in mitmproxy. [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170213152727.png but it is still empty after run dynamic analyze, does this means the vuln analyzed by static analyzer is not realy existed? thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-279577029, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qg1A_Z4ngPKrJ_273eSbntXGH7uJks5rcQCHgaJpZM4LBHMD .

cnscyy commented 7 years ago

yes, it is exactly like you say. after run the dynamic test, three file will be created in client/. they are com.changhong.infosec.safecamera_files、com.changhong.infosec.safecamera_network_traffic、com.changhong.infosec.safecamera.apk. com.changhong.infosec.safecamera_network_traffic is empty like you say. what should i do? is this because of lack of some certificates?

jheguia commented 7 years ago

Can you check https://play.google.com/store/apps/details?id=ar.gob.buenosaires.reclamos? The static analyzer found SSL_WEBVIEW_ERROR, PHONEGAP_JS_INJECTION and JAVASCRIPTINTERFACE on it, and the dynamic analyzer managed to verify all of them (although it took very many tries for PHONEGAP_JS and JAVASCRIPTINTERFACE). If you check it for the latter two, even if it doesn't trigger the vuln you should see some URLS being visited and some traffic being stored in the _network_traffic file. If not, there's probably a problem with mitmproxy. Also, the SSL_WEBVIEW_ERROR should be verifiable in very few tries.

Cheers, Juan

2017-02-14 0:27 GMT-03:00 cnscyy notifications@github.com:

yes, it is exactly like you say. after run the dynamic test, three file will be created in client/. they are com.changhong.infosec.safecamera_files、com. changhong.infosec.safecamera_network_traffic、com.changhong.infosec.safecamera.apk. com.changhong.infosec.safecamera_network_traffic is empty like you say. what should i do? is this because of lack of some certificates?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-279597539, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qgRSzilJp1sqNdzzDH7e4PHRYVusks5rcR8OgaJpZM4LBHMD .

cnscyy commented 7 years ago

when i test this App by static test, the results is: image after dynamic test, it is still empty in ar.gob.buenosaires.reclamos_network_traffic. and the result of test is also empty. what is the problem in mitmproxy? i did not install correctly?

thanks again!

jheguia commented 7 years ago

I believe libmproxy could be the problem, yes. Let's do this: open a terminal, run a Python interpreter (I like ipython) and do:

from libmproxy import version print version.NAMEVERSION

Tell me if the NAMEVERSION is 'mitmproxy 0.11.3'

Cheers, Juan

2017-02-15 7:03 GMT-03:00 cnscyy notifications@github.com:

when i test this App by static test, the results is: [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170215173257.png after dynamic test, it is still empty in ar.gob.buenosaires.reclamos_network_traffic. and the result of test is also empty. what is the problem in mitmproxy? i did not install correctly?

thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-279968187, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qkk2tg1LgxHJG975WiYBNclCChDfks5rcs12gaJpZM4LBHMD .

cnscyy commented 7 years ago

hi, the NAMEVERSION is 'mitmproxy 0.11.3' image i download the mitmproxy setup program from:github

thanks again!

jheguia commented 7 years ago

I have just pushed some changes into the repository, mainly logging. Everything will go into /tmp/VMClient.log. Also there was this bit in VMClient.py where I had to cast a string into int, which is the only real behavior change but might be crucial.

Cheers, Juan

2017-02-15 23:24 GMT-03:00 cnscyy notifications@github.com:

hi, the NAMEVERSION is 'mitmproxy 0.11.3' [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170216101709.png i download the mitmproxy setup program from:github https://github.com/mitmproxy/mitmproxy/releases/tag/v0.13

thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-280210661, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9quKna06sHvhBpx7uQnGezJiYQz4fks5rc7NogaJpZM4LBHMD .

cnscyy commented 7 years ago

hi, i have test the new project. the result of the test is in VMclient.log

thanks again!

jheguia commented 7 years ago

I see that there's a difference between your logs and mine where the program tries to set the default gateway and gets a "File exists" error. It might be nothing, but it is worth checking the routes are correct.The monkey program is invoked too, but the traffic is missing. If you look at the emulator screen when the monkey runs, does it do anything? If the monkey does exercise the program, then it should be either the routing or the proxy. What version of netlib does Python see? Check netlib.version.NAMEVERSION: it should be 'netlib 0.11.2'

Another thing you could try is checking with tcpdump that the traffic from the emulator is reaching the machine where VMClient runs.

Cheers, Juan

2017-02-17 1:37 GMT-03:00 cnscyy notifications@github.com:

hi, i have test the new project. the result of the test is in VMclient.log https://raw.githubusercontent.com/cnscyy/image/master/image/VMClient.log

thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-280549814, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qmM0amzOwpV9-HCX45s2jC4LLIIiks5rdSQDgaJpZM4LBHMD .

cnscyy commented 7 years ago

hi, thanks again! for the first time to set the default gateway, it is ok. then the second time or third time, it will get the error "route: SIOCADDRT: File exists". image i also check the version of netlib, it is 0.11.2 image when running the monkey test, the emulator will test the apk. image

when i use tcpdump to capture the traffic of client and emulators, the results show that there is problem in tcp communication. there is a question of out-of order about the packets, and always retransmit the data. image maybe this is the problem. now i haven't find the solution. the route table of before/after set the gateway is: image

thanks again!

cnscyy commented 7 years ago

i have set the emulator and client in a Intranet. the tcp communication become OK! the problem of out-of order disappear. image but there is still the problem. the result of analyzer is empty, the _network_traffic file is also empty.

thanks!

jheguia commented 7 years ago

I see you end up with two default gateways: that should not be, but if you get traffic it should be analyzed. Do you see the app being exercised when the client runs? How long does it run?

2017-02-20 7:23 GMT-03:00 cnscyy notifications@github.com:

i have set the emulator and client in a Intranet. the tcp communication become OK! the problem of out-of order disappear. [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170220181805.png but there is still the problem. the result of analyzer is empty, the _network_traffic file is also empty.

thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-281041216, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qpxE4n-nmHeHeE42hokv6UccEpYaks5reWmqgaJpZM4LBHMD .

cnscyy commented 7 years ago

i can see the app being exercised when run the monkey cmd. but it will do the exercised again and again!

cnscyy commented 7 years ago

could you give me more apps which can be test in the dynamic analyze? thanks again!

jheguia commented 7 years ago

https://play.google.com/store/apps/details?id=com.fatsecret.android should test positive for SSL_CUSTOM_HOSTNAMEVERIFIER and SSL_CUSTOM_TRUSTMANAGER almost immediately.

2017-02-20 23:47 GMT-03:00 cnscyy notifications@github.com:

could you give me more apps which can be test in the dynamic analyze? thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-281230414, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qj50DAQBgM4xWSbYw7iMUEkVocx-ks5relA3gaJpZM4LBHMD .

cnscyy commented 7 years ago

hello, i find that when i install the emulator, run the cmd:adb -s "$emulator" shell "pm uninstall com.google.android.gms", the result show failed. is this will effect the dynamic analyze?

thanks again!

jheguia commented 7 years ago

I don't think so. Today I installed the dynamic analyzer from scratch and it worked (on debug mode at least). Since netlib and mitmproxy have to be installed from old versions, I had to get their dependencies by hand for their setup scripts to work. What I did was download the .tar.gz files for netlib 0.11.2 and mitmproxy 0.11.3 from the mitmproxy site, and git clone the Marvin-dynamic-Analyzer repo. I ended up writing a script to get the dependencies, which I'm attaching (it works for Ubuntu, should have no problem in Debian). The script will download the required versions of mitmproxy and netlib, install the requirements, and run the setup scripts for both of them. After that, if you clone the repository, run client_setup.sh (this has to be done after each boot) and it should run in test mode with some minimal configuration (such as setting a route for the client host in the emulator and then the GW). Maybe you could try this in a fresh VM? Of course it's just for the dynamic analyzer client, but this way you should be able to capture traffic.

Cheers, Juan

2017-02-22 10:58 GMT-03:00 cnscyy notifications@github.com:

hello, i find that when i install the emulator, run the cmd:adb -s "$emulator" shell "pm uninstall com.google.android.gms", the result show failed. is this will effect the dynamic analyze?

thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-281676103, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qrhiN57m6r3xKR33Uz1VkTpSfUY3ks5rfD7vgaJpZM4LBHMD .

cnscyy commented 7 years ago

hello, i haven't see the link for teh script in the project. would you forget to attck the script?

thanks again for your patient.

jheguia commented 7 years ago

Sorry, it seems Github didn't like the attachment in the message; I committed the script now. It's in Marvin-dynamic-Analyzer/client/mitmproxy_setup.sh

Cheers, Juan

El 24 feb. 2017 00:53, "cnscyy" notifications@github.com escribió:

hello, i haven't see the link for teh script in the project. would you forget to attck the script?

thanks again for your patient.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-282195099, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qoudnIEGlb87_V6FR3xfqSWoC2fSks5rflAZgaJpZM4LBHMD .

cnscyy commented 7 years ago

hello, i fresh all the emulators and set the ip and routes manually. image i also reinstall the mitmproxy and netlib using the script. i set the debug = True, and use the cmd: python VMClient. SSL_WEBVIEW_ERROR ar.gob.buenosaires.reclamos '''{"emulator":"192.168.137.102","count":0}''' to test on debug mode. but it still empty in the _network_traffic file. the result is also empty.

i fount tha the function handle_request and handle_response in Analyzer.py is not be called.

i build the environment using virtualbox. i create the client and emulators in intnet mode and set the ip and routes manually. they can ping properly. is there some difference between the environment of mine and yours?

thanks again!

jheguia commented 7 years ago

Hi, That app might not always have traffic, it depends on what the fuzzer does. It's better to try com.fatsecret.android, that app always has traffic. I asked you to try a fresh VM so as to be sure it's not the analyzer but (perhaps, somehow) the client, so if you can make a fresh one it'll help to make sure everything is like on our setup. I have the client running in OpenNebula, but installed from scratch in a clean Ubuntu 14.04 VM in VirtualBox and it ran well. Of course, always remember to run client_setup.sh after booting, otherwise you won't see the traffic and everything else will be normal. Just to be sure, the program is supposed to run with Python 2.7.3.

Cheers, Juan

2017-03-01 7:20 GMT-03:00 cnscyy notifications@github.com:

hello, i fresh all the emulators and set the ip and routes manually. [image: image] https://raw.githubusercontent.com/cnscyy/image/master/image/20170301173337.png i also reinstall the mitmproxy and netlib using the script. i set the debug = True, and use the cmd: python VMClient. SSL_WEBVIEW_ERROR ar.gob.buenosaires.reclamos '''{"emulator":"192.168.137.102","count":0}''' to test on debug mode. but it still empty in the _network_traffic file. the result is also empty.

i fount tha the function handle_request and handle_response in Analyzer.py is not be called.

i build the environment using virtualbox. i create the client and emulators in intnet mode and set the ip and routes manually. they can ping properly. is there some difference between the environment of mine and yours?

thanks again!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/programa-stic/Marvin-dynamic-Analyzer/issues/1#issuecomment-283301395, or mute the thread https://github.com/notifications/unsubscribe-auth/AIB9qqLft4yPStheKK-2a0gWoXjP_v-Kks5rhUZmgaJpZM4LBHMD .