search

MobSF / Mobile-Security-Framework-MobSF

Mobile Security Framework (MobSF) is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing, malware analysis and security assessment framework capable of performing static and dynamic analysis.
https://opensecurity.in
GNU General Public License v3.0
17.29k stars 3.22k forks source link

Dynamic analysis fails despite proper configuration #570

Closed Kollerb04 closed 6 years ago

Kollerb04 commented 6 years ago

I have configured the virtual machine in virtualbox 1:1 with https://github.com/MobSF/Mobile-Security-Framework-MobSF/wiki/1.-Documentation

The mobsf/settings.py contains:

UUID = '<uuid from vbox file>'
SUUID = '<suuid taken from current snapshot from the vbox file>'
VM_IP= '192.168.56.3'
VM_ADB_PORT = 5555

PROXY_IP = '192.168.56.1'
PORT = 1337

The proxy ip is the adress of the vbox ip adapter thats assigned to the vm machine But all I'm getting is:

Could not find a registered machine with UUID {uuid from vbox file}
Details" code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001) ... 
Context: "FindMachine(Bstr(a->...
ajinabraham commented 6 years ago

Double check the UUID. This error comes when the VM with given UUID is not found.

Kollerb04 commented 6 years ago

I did, its copied directly from the MobSD_VM_0.2.vbox, the output log also displays the same uuid

ajinabraham commented 6 years ago

Can you share your vbox file and settings.py

Kollerb04 commented 6 years ago 
"""
Django settings for MobSF project.

For more information on this file, see
https://docs.djangoproject.com/en/dev/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/dev/ref/settings/
"""

import os
import imp
from MobSF import utils

from install.windows.setup import windows_config_local

#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
#       MOBSF FRAMEWORK CONFIGURATIONS
#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
#==============================================

MOBSF_VER = "v1.0 Beta"
BANNER = """
  __  __       _    ____  _____         _   ___  
 |  \/  | ___ | |__/ ___||  ___| __   _/ | / _ \ 
 | |\/| |/ _ \| '_ \___ \| |_    \ \ / / || | | |
 | |  | | (_) | |_) |__) |  _|    \ V /| || |_| |
 |_|  |_|\___/|_.__/____/|_|       \_/ |_(_)___/ 

"""
# ASCII Standard
#==============================================

#==========MobSF Home Directory=================
USE_HOME = False

# True : All Uploads/Downloads will be stored in user's home directory
# False : All Uploads/Downloads will be stored in MobSF root directory
# If you need multiple users to share the scan results set this to False
#===============================================

MobSF_HOME = utils.getMobSFHome(USE_HOME)
# Logs Directory
LOG_DIR = os.path.join(MobSF_HOME, 'logs/')
# Download Directory
DWD_DIR = os.path.join(MobSF_HOME, 'downloads/')
# Screenshot Directory
SCREEN_DIR = os.path.join(MobSF_HOME, 'downloads/screen/')
# Upload Directory
UPLD_DIR = os.path.join(MobSF_HOME, 'uploads/')
# Database Directory
DB_DIR = os.path.join(MobSF_HOME, 'db.sqlite3')

# Database
# https://docs.djangoproject.com/en/dev/ref/settings/#databases

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': DB_DIR,
    }
}
# Postgres DB - Install psycopg2
'''
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'mobsf',
        'USER': 'postgres',
        'PASSWORD': '',
        'HOST': 'localhost',
        'PORT': '',
    }
}
'''
#===============================================

#==========LOAD CONFIG FROM MobSF HOME==========
try:
    # Update Config from MobSF Home Directory
    if USE_HOME:
        USER_CONFIG = os.path.join(MobSF_HOME, 'config.py')
        sett = imp.load_source('user_settings', USER_CONFIG)
        locals().update(
            {k: v for k, v in list(sett.__dict__.items()) if not k.startswith("__")})
        CONFIG_HOME = True
    else:
        CONFIG_HOME = False
except:
    utils.PrintException("[ERROR] Parsing Config")
    CONFIG_HOME = False
#===============================================

#=============ALLOWED EXTENSIONS================
ALLOWED_EXTENSIONS = {
    ".txt": "text/plain",
    ".png": "image/png",
    ".zip": "application/zip",
    ".tar": "application/x-tar"
}
#===============================================

#=============ALLOWED MIMETYPES=================

APK_MIME = [
    'application/octet-stream',
    'application/vnd.android.package-archive',
    'application/x-zip-compressed',
    'binary/octet-stream',
]
IPA_MIME = [
    'application/iphone',
    'application/octet-stream',
    'application/x-itunes-ipa',
    'application/x-zip-compressed',
    'binary/octet-stream',
]
ZIP_MIME = [
    'application/zip',
    'application/octet-stream',
    'application/x-zip-compressed',
    'binary/octet-stream',
]
APPX_MIME = [
    'application/octet-stream',
    'application/vns.ms-appx',
    'application/x-zip-compressed'
]

#===============================================

#=====MOBSF SECRET GENERATION AND MIGRATION=====
# Based on https://gist.github.com/ndarville/3452907#file-secret-key-gen-py
try:
    SECRET_KEY
except NameError:
    SECRET_FILE = os.path.join(MobSF_HOME, "secret")
    try:
        SECRET_KEY = open(SECRET_FILE).read().strip()
    except IOError:
        try:
            SECRET_KEY = utils.genRandom()
            secret = open(SECRET_FILE, 'w')
            secret.write(SECRET_KEY)
            secret.close()
        except IOError:
            Exception('Please create a %s file with random characters \
            to generate your secret key!' % SECRET_FILE)
        # Run Once
        # Windows Setup
        windows_config_local(MobSF_HOME)
        utils.make_migrations(BASE_DIR)
        utils.migrate(BASE_DIR)
        utils.kali_fix(BASE_DIR)

#=============================================

#============DJANGO SETTINGS =================

# SECURITY WARNING: don't run with debug turned on in production!
# ^ This is fine Do not turn it off until MobSF moves from Beta to Stable

DEBUG = True
ALLOWED_HOSTS = ['127.0.0.1', 'testserver', '*']
# Application definition
INSTALLED_APPS = (
    #'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'StaticAnalyzer',
    'DynamicAnalyzer',
    'MobSF',
    'MalwareAnalyzer',
)
MIDDLEWARE_CLASSES = (
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'django.middleware.security.SecurityMiddleware',
)
ROOT_URLCONF = 'MobSF.urls'
WSGI_APPLICATION = 'MobSF.wsgi.application'
# Internationalization
# https://docs.djangoproject.com/en/dev/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
MEDIA_ROOT = os.path.join(BASE_DIR, 'uploads')
MEDIA_URL = '/uploads/'
TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'APP_DIRS': True,
        'DIRS':
            [
                os.path.join(BASE_DIR, 'templates')
            ],
        'OPTIONS':
            {
                'debug': True,
            }
    },
]
STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'static/'),
)
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/dev/howto/static-files/
STATIC_URL = '/static/'

#===================
# USER CONFIGURATION
#===================

if CONFIG_HOME:
    print("[INFO] Loading User config from: " + USER_CONFIG)
else:
    '''
    IMPORTANT
    If 'USE_HOME' is set to True, then below user configuration settings are not considered.
    The user configuration will be loaded from config.py in MobSF Home directory.
    '''
    #^CONFIG-START^: Do not edit this line
    #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    #          MOBSF USER CONFIGURATIONS
    #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    #-------------------------
    # STATIC ANALYZER SETTINGS
    #-------------------------

    #==========ANDROID SKIP CLASSES==========================
    # Common third party classes that will be skipped during static analysis
    SKIP_CLASSES = [
        r'android[\\\/]{1}support[\\\/]{1}', r'com[\\\/]{1}google[\\\/]{1}', r'android[\\\/]{1}content[\\\/]{1}',
        r'com[\\\/]{1}android[\\\/]{1}', r'com[\\\/]{1}facebook[\\\/]{1}', r'com[\\\/]{1}twitter[\\\/]{1}',
        r'twitter4j[\\\/]{1}', r'org[\\\/]{1}apache[\\\/]{1}', r'com[\\\/]{1}squareup[\\\/]{1}okhttp[\\\/]{1}',
        r'oauth[\\\/]{1}signpost[\\\/]{1}', r'org[\\\/]{1}chromium[\\\/]{1}'
    ]

    #==========DECOMPILER SETTINGS=================

    DECOMPILER = "cfr"

    # Three Decompilers are available
    # 1. jd-core
    # 2. cfr
    # 3. procyon

    #==============================================

    #==========Dex to Jar Converter================
    JAR_CONVERTER = "d2j"

    # Two Dex to Jar converters are available
    # 1. d2j
    # 2. enjarify
    #==============================================

    #======WINDOWS STATIC ANALYSIS SETTINGS ===========

    # Private key
    WINDOWS_VM_SECRET = 'MobSF/windows_vm_priv_key.asc'
    # IP and Port of the MobSF Windows VM
    # eg: WINDOWS_VM_IP = '127.0.0.1'
    WINDOWS_VM_IP = None
    WINDOWS_VM_PORT = '8000'
    #==================================================

    #==============3rd Party Tools=================
    '''
    If you want to use a different version of 3rd party tools used by MobSF.
    You can do that by specifying the path here. If specified, MobSF will run
    the tool from this location.
    '''

    # Android 3P Tools
    DEX2JAR_BINARY = ""
    BACKSMALI_BINARY = ""
    CFR_DECOMPILER_BINARY = ""
    JD_CORE_DECOMPILER_BINARY = ""
    PROCYON_DECOMPILER_BINARY = ""
    APKTOOL_BINARY = ""
    ADB_BINARY = ""
    ENJARIFY_DIRECTORY = ""

    # iOS 3P Tools
    OTOOL_BINARY = ""
    CLASSDUMPZ_BINARY = ""

    # COMMON
    JAVA_DIRECTORY = ""
    VBOXMANAGE_BINARY = ""
    PYTHON3_PATH = ""

    '''
    Examples:
    JAVA_DIRECTORY = "C:/Program Files/Java/jdk1.7.0_17/bin/"
    JAVA_DIRECTORY = "/usr/bin/"
    DEX2JAR_BINARY = "/Users/ajin/dex2jar/d2j-dex2jar.sh"
    ENJARIFY_DIRECTORY = "D:/enjarify/"
    VBOXMANAGE_BINARY = "/usr/bin/VBoxManage"
    CFR_DECOMPILER_BINARY = "/home/ajin/tools/cfr.jar"
    PYTHON3_PATH = "C:/Users/Ajin/AppData/Local/Programs/Python/Python35-32/"
    '''
    #===============================================

    #-------------------------
    # DYNAMIC ANALYZER SETTINGS
    #-------------------------

    #========ANDROID DYNAMIC ANALYSIS SETTINGS================================

    ANDROID_DYNAMIC_ANALYZER = "MobSF_VM"

    # You can choose any of the below
    # 1. MobSF_VM
    # 2. MobSF_AVD
    # 3. MobSF_REAL_DEVICE

    '''
    MobSF_VM - x86 Android 4.4.2 running on VirtualBox (Fast, not all Apps work)
    MobSF_AVD - ARM Android 4.1.2 running on Android Emulator (Slow, Most Apps work)
    MobSF_REAL_DEVICE - Rooted Android 4.03 - 4.4 Device (Very Fast, All Apps work)
    Supports Android 5+ for real device. Not tested!
    '''

    #=========================================================================

    #=======ANDROID REAL DEVICE SETTINGS===========
    DEVICE_IP = '192.168.1.18'
    DEVICE_ADB_PORT = 5555
    DEVICE_TIMEOUT = 300
    #==============================================

    #===========ANDROID EMULATOR SETTINGS ===========
    # generated by mobsfy_AVD.py, do not edit the
    # below AVD settings yourself.
    AVD_EMULATOR = "avd_emulator"
    AVD_PATH = "avd_path"
    AVD_REFERENCE_NAME = r'Nexus5API16'
    AVD_DUP_NAME = r'Nexus5API16_1'
    AVD_ADB_PORT = 5554
    #================================================

    #====ANDROID MOBSF VIRTUALBOX VM SETTINGS =====
    # VM UUID
    UUID = '8f192884-14df-458e-995d-b37dae0d2390'
    # Snapshot UUID
    SUUID = 'c10e0114-a5dc-4631-8f54-91c010968734'
    # IP of the MobSF VM
    VM_IP = '192.168.56.3'
    VM_ADB_PORT = 5555
    VM_TIMEOUT = 100
    #==============================================

    #--------------------------
    # MobSF MITM PROXY SETTINGS
    #--------------------------

    #================HOST/PROXY SETTINGS ===============
    PROXY_IP = '192.168.56.1'  # Host/Server/Proxy IP
    PORT = 1337  # Proxy Port
    ROOT_CA = '0026aabb.0'
    SCREEN_IP = PROXY_IP  # ScreenCast IP
    SCREEN_PORT = 9339  # ScreenCast Port(Do not Change)
    #===================================================

    #========UPSTREAM PROXY SETTINGS ==============
    # If you are behind a Proxy
    UPSTREAM_PROXY_ENABLED = False
    UPSTREAM_PROXY_SSL_VERIFY = True
    UPSTREAM_PROXY_TYPE = "http"
    UPSTREAM_PROXY_IP = "127.0.0.1"
    UPSTREAM_PROXY_PORT = 3128
    UPSTREAM_PROXY_USERNAME = ""
    UPSTREAM_PROXY_PASSWORD = ""
    #==============================================

    #--------------------------
    # MALWARE ANALYZER SETTINGS
    #--------------------------

    DOMAIN_MALWARE_SCAN = True

    #========DISABLED COMPONENTS===================

    #----------VirusTotal--------------------------
    VT_ENABLED = False
    VT_API_KEY = 'XXXXXXXXXXXXXX'
    VT_UPLOAD = False
    # Before setting VT_ENABLED to True,
    # Make sure VT_API_KEY is set to your VirusTotal API key
    # register at: https://www.virustotal.com/#/join-us
    # You can get your API KEY from https://www.virustotal.com/en/user/<username>/apikey/
    # VT has a premium features but the free account is just enough for personal use
    # BE AWARE - if you enable VT, in case the file wasn't already uploaded to VirusTotal,
    # It will be uploaded if you set VT_UPLOAD to True!
    #==============================================

    #----------APKiD-------------------------------
    APKID_ENABLED = False
    # Before setting APKID_ENABLED to True,
    # Install rednaga fork of Yara Python
    # git clone https://github.com/rednaga/yara-python
    # cd yara-python
    # python setup.py install
    #==============================================

    #^CONFIG-END^: Do not edit this line

# The below code should be loaded last.
#============JAVA SETTINGS======================
JAVA_PATH = utils.FindJava(False)
#===============================================

#================VirtualBox Settings============
VBOX = utils.FindVbox(False)
#===============================================
Kollerb04 commented 6 years ago 
<?xml version="1.0"?>
<!--
** DO NOT EDIT THIS FILE.
** If you make changes to this file while any VirtualBox related application
** is running, your changes will be overwritten later, without taking effect.
** Use VBoxManage or the VirtualBox Manager GUI to make changes.
-->
<VirtualBox xmlns="http://www.virtualbox.org/" version="1.12-macosx">
  <Machine uuid="{8f192884-14df-458e-995d-b37dae0d2390}" name="MobSF_VM_0.2" OSType="Linux" currentSnapshot="{c10e0114-a5dc-4631-8f54-91c010968734}" snapshotFolder="Snapshots" lastStateChange="2018-03-21T07:59:15Z">
    <Description>MobSF Dynamic Analysis VM</Description>
    <MediaRegistry>
      <HardDisks>
        <HardDisk uuid="{f14ddc3b-89ad-4557-a7da-fe8ae05f115d}" location="MobSF_VM_0.3-disk1.vmdk" format="VMDK" type="Normal">
          <HardDisk uuid="{0283aa52-cfcd-4eff-ac7e-a20bfc978be5}" location="Snapshots/{0283aa52-cfcd-4eff-ac7e-a20bfc978be5}.vmdk" format="VMDK"/>
          <HardDisk uuid="{de7e145c-5e2a-40b6-9090-fc9af5159d8c}" location="Snapshots/{de7e145c-5e2a-40b6-9090-fc9af5159d8c}.vmdk" format="VMDK"/>
 ...
ajinabraham commented 6 years ago

Can you manually run this from VirtualBox directory

VBoxManage snapshot <uuid> restore <snapshot_uuid> and let me know about the result

Kollerb04 commented 6 years ago

It does say:

Restoring snapshot 'SAVED_STATE' (c10e...)
0%...10%...100%

but there is no difference with the dynamic scan

ajinabraham commented 6 years ago

So does the command restore the VM?

Kollerb04 commented 6 years ago

Yes, I even deleted the snapshot and created a new one but unfortunately the error message stays the same as it seems to be crashing while performing "Refreshing MobSF VM" "VM Closed" and "VM Restore Snapshot" but the log mentions only the UUID not the SUUID. All this happens when I enter the "Start dynamic analysis" panel, thats when these error messages pop in. When I click on "create environment" it crashes saying that capfuzz file or directory is missing which is false.

When I run VBoxManage list vms I receive the correct name and UUID

ajinabraham commented 6 years ago

Looks like your installation is not correct. It shouldn't complain about CapFuzz. Can you try installing the latest master once again?

ajinabraham commented 6 years ago

Also if you still having issue. Do sent a mail at ajin25 AT gmail DOT com. Let's debug the issue.

Kollerb04 commented 6 years ago

Just did. No error messages during install yet still the same error during dynamic scan.

blazma0707 commented 6 years ago

I've solved this problem on the same situation. I manually registered VM like below. And then problem solved.

VBoxManage registervm /MobSF_VM/MobSF_VM.vbox

ajinabraham commented 6 years ago

@Kollerb04 Can you try the above once? @blazma0707 Thanks for the lead!

Kollerb04 commented 6 years ago

Actually this is pretty interesting. If I understood correctly that means my standard path to the image, not setting explicit naming like /MobSF_VM/ itself.

So I have tried the command and weird results came in. My path is /myuser/VirtualBox VMs/MobSF_VM_0.2

But when I run that command with /VirtualBox\ VMs/... it says VBoxManage: error: Runtime error opening '/Users/myuser/Library/VirtualBox/MobSF_VM_0.2.vbox' for reading: -102 (File not found.). VBoxManage: error: /Users/vbox/tinderbox/5.2-mac-rel/src/VBox/Main/src-server/MachineImpl.cpp[479] (nsresult Machine::initFromSettings(VirtualBox *, const com::Utf8Str &, const com::Guid *)) VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine, callee nsISupports VBoxManage: error: Context: "OpenMachine(Bstr(a->argv[0]).raw(), machine.asOutParam())" at line 87 of file VBoxManageMisc.cpp

When I paste the virtual machine file into the folder I get: VBoxManage registervm MobSF_VM_0.2.vbox VBoxManage: error: Trying to open a VM config 'MobSF_VM_0.2.vbox' which has the same UUID as an existing virtual machine VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine, callee nsISupports VBoxManage: error: Context: "OpenMachine(Bstr(a->argv[0]).raw(), machine.asOutParam())" at line 87 of file VBoxManageMisc.cpp

HILALUDEEN commented 6 years ago

Hi, [INFO] Dynamic Analysis Started

[INFO] Refreshing MobSF VM [ERROR] VirtualBox not found, Manually set VBOXMANAGE_BINARY in settings.py

[2018-04-12 10:59:34] [ERROR] Refreshing MobSF VM (/root/Mobile-Security-Framework-MobSF/DynamicAnalyzer/views/android/virtualbox_vm.py, LINE 14 "subprocess.call(args)"): expected str, bytes or os.PathLike object, not NoneType

ajinabraham commented 6 years ago

@HILALUDEEN VirtualBox not found, !! Manually set VBOXMANAGE_BINARY in settings.py Read errors carefully!

ajinabraham commented 6 years ago

@Kollerb04 This looks to me like some virtualbox issue in your box, try with latest version.

HILALUDEEN commented 6 years ago

@ajinabraham What value i should pass in the parameter VBOXMANAGE_BINARY...??

Kollerb04 commented 6 years ago

@ajinabraham with the latest version of the VM? I just did redownloaded and added 1:1 following the instructions in the documentation (I'm using mac) and theres exact the same issue happening. Virtualbox version is 5.2.8 r121009

ajinabraham commented 6 years ago

@HILALUDEEN Path to the VBoxManage binary.

@Kollerb04 We are using VirtualBox's vboxmange to invoke the VMs registered with virtualbox and I think this issue has to do with your VirtualBox installation being not able to register and run the VM. It's out of scope with the project. You might have to check with VirtualBox project maintainers and troubleshoot the issue.

HILALUDEEN commented 6 years ago

Hi,

How to update google play services in VM

On Fri, Apr 13, 2018 at 3:26 PM Ajin Abraham notifications@github.com wrote:

@HILALUDEEN https://github.com/HILALUDEEN Path to the VBoxManage binary.

@Kollerb04 https://github.com/Kollerb04 We are using VirtualBox's vboxmange to invoke the VMs registered with virtualbox and I think this issue has to do with your VirtualBox installation being not able to register and run the VM. It's out of scope with the project. You might have to check with VirtualBox project maintainers and troubleshoot the issue.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/MobSF/Mobile-Security-Framework-MobSF/issues/570#issuecomment-381087090, or mute the thread https://github.com/notifications/unsubscribe-auth/Akgp5eRgVZw37v-_Fw_7Cg8n__XrX-fKks5toHZogaJpZM4S2n7s .

abhinav13shukla commented 6 years ago

Hi Ajin, I am facing the same issue in when I start the Dynamic analysis. I tried several methods and I also try using a lower version of VirtualBox but it is still not working. I am using MacOs.