search

WithSecureLabs / needle

The iOS Security Testing Framework
https://mobiletools.mwrinfosecurity.com/
Other
1.34k stars 284 forks source link

dependency_installer does not work #268

Closed HanseSecure closed 5 years ago

HanseSecure commented 5 years ago

Issue

Expected behaviour

Install all dependencies on the iOS device via dependency_installer

Actual behaviour

several error messages (see error log) && modules do not work

Also tried solutions from #172

needle error logs

[] Checking connection with device... [+] Already connected to: 192.168.178.28 [D] Creating temp folder: /var/root/needle/ [D] [REMOTE CMD] Remote Command: if [ -d /var/root/needle/ ]; then echo "yes"; else echo "no" ; fi [] Checking prerequisites... [D] [REMOTE CMD] Remote Command: which apt-get [D] [REMOTE CMD] Remote Command: which dpkg [D] [INSTALL] Already installed: coreutils. [D] [INSTALL] Already installed: coreutils-bin. [D] [REMOTE CMD] Remote Command: apt-get update [!] W: Failed to fetch https://build.frida.re/./Packages.gz Sub-process gzip received signal 9. // skipped repeated error messages [?] Trying to continue anyway... [D] [REMOTE CMD] Remote Command: cat /etc/apt/sources.list.d/cydia.list [*] The following tools are going to be installed: ['FILEDP', 'KEYCHAIN_DUMP', 'CLASS-DUMP', 'ONDEVICECONSOLE', 'PLUTIL', 'GDB', 'CYCRIPT', 'CURL', 'OPEN', 'SCP', 'PERL', 'COREUTILS', 'FRIDA', 'FRIDA32BIT', 'UNZIP', 'FSMON', 'DARWINTOOLS', 'GAWK', 'THEOS', 'CLUTCH', 'PBWATCHER']

[D] [REMOTE CMD] Remote Command: which FileDP [D] [INSTALL] Tool already available: FILEDP [D] [REMOTE CMD] Remote Command: which keychain_dump [D] [INSTALL] Tool already available: KEYCHAIN_DUMP [D] [INSTALL] Already installed: pcre. [D] [INSTALL] Already installed: net.limneos.classdump-dyld. [D] [INSTALL] Already installed: class-dump. [D] [INSTALL] Already installed: com.eswick.ondeviceconsole. [D] [INSTALL] Already installed: com.ericasadun.utilities. [D] [INSTALL] Already installed: gdb. [D] [INSTALL] Already installed: cycript. [D] [INSTALL] Already installed: curl. [D] [INSTALL] Already installed: com.conradkramer.open. [D] [INSTALL] Already installed: org.coolstar.scp-sftp-dropbear. [V] [INSTALL] Installing PERL via apt-get. [D] Adding repo to cydia.list: http://coolstar.org/publicrepo/ [D] [REMOTE CMD] Remote Command: echo "deb http://coolstar.org/publicrepo/ ./" >> /etc/apt/sources.list.d/cydia.list [D] [REMOTE CMD] Remote Command: apt-get update [!] W: Failed to fetch https://build.frida.re/./Packages.gz Sub-process gzip received signal 9. // skipped repeated error messages [?] Trying to continue anyway... [D] [REMOTE CMD] Remote Command: apt-get install -y --force-yes org.coolstar.perl [!] dpkg: error processing archive /var/cache/apt/archives/org.coolstar.perl_5.22.0_iphoneos-arm.deb (--unpack): cannot copy extracted data for './usr/local/bin/perl' to '/usr/local/bin/perl.dpkg-new': failed to write (No space left on device) dpkg-deb: error: subprocess paste was killed by signal (Broken pipe: 13) Errors were encountered while processing: /var/cache/apt/archives/org.coolstar.perl_5.22.0_iphoneos-arm.deb E: Sub-process /usr/bin/dpkg returned an error code (1) [V] [SSH] Disconnecting... [V] [AGENT] Disconnecting from agent... [?] Resetting connection to device... [V] [AGENT] Connecting to agent (192.168.178.28:4444)... [?] Error occurred during apt-get update: [?] Trying to continue anyway... [D] [REMOTE CMD] Remote Command: apt-get install -y --force-yes re.frida.server [!] 'NoneType' object has no attribute 'open_session' [V] [SSH] Disconnecting... [V] [AGENT] Disconnecting from agent... [?] Resetting connection to device... [V] [AGENT] Connecting to agent (192.168.178.28:4444)... [?] Error occurred during installation of tools: [?] Trying to continue anyway... [V] [INSTALL] Installing FRIDA32BIT via apt-get. [D] Adding repo to cydia.list: https://build.frida.re/ [D] [REMOTE CMD] Remote Command: echo "deb https://build.frida.re/ ./" >> /etc/apt/sources.list.d/cydia.list [?] Error occurred while adding a new repo: An error occurred and it was not possible to restore it (3 attempts failed) [?] Trying to continue anyway... [D] [REMOTE CMD] Remote Command: apt-get install -y --force-yes re.frida.server32 [!] 'NoneType' object has no attribute 'open_session' [V] [SSH] Disconnecting... [V] [AGENT] Disconnecting from agent... [?] Resetting connection to device... [V] [AGENT] Connecting to agent (192.168.178.28:4444)... [?] Error occurred during installation of tools: [?] Trying to continue anyway... [D] [INSTALL] Already installed: unzip. [D] [REMOTE CMD] Remote Command: which fsmon [!] 'NoneType' object has no attribute 'open_session' [V] [SSH] Disconnecting... [V] [AGENT] Disconnecting from agent... [?] Resetting connection to device... [V] [AGENT] Connecting to agent (192.168.178.28:4444)... [?] Error occurred during installation of tools: [?] Trying to continue anyway... [V] [INSTALL] Installing DARWINTOOLS via apt-get. [D] [REMOTE CMD] Remote Command: apt-get install -y --force-yes org.coolstar.cctools [!] 'NoneType' object has no attribute 'open_session' [V] [SSH] Disconnecting... [V] [AGENT] Disconnecting from agent... [?] Resetting connection to device... [V] [AGENT] Connecting to agent (192.168.178.28:4444)... [?] Error occurred during installation of tools: [?] Trying to continue anyway... [D] [INSTALL] Already installed: gawk. [V] [INSTALL] Manually installing: THEOS [D] [REMOTE CMD] Remote Command: ln -s /usr/local/bin/perl /usr/bin/perl [?] Error occurred during installation of tools: An error occurred and it was not possible to restore it (3 attempts failed) [?] Trying to continue anyway... [V] [INSTALL] Manually installing: CLUTCH [D] [REMOTE CMD] Remote Command: curl -ksL "http://cydia.iphonecake.com/Clutch2.0.4.deb" -o /var/root/clutch.deb [!] 'NoneType' object has no attribute 'open_session' [V] [SSH] Disconnecting... [V] [AGENT] Disconnecting from agent... [?] Resetting connection to device... [V] [AGENT] Connecting to agent (192.168.178.28:4444)... [?] Error occurred during installation of tools: [?] Trying to continue anyway... [D] [REMOTE CMD] Remote Command: which pbwatcher [!] 'NoneType' object has no attribute 'open_session' V] [SSH] Disconnecting... [V] [AGENT] Disconnecting from agent... [?] Resetting connection to device... [V] [AGENT] Connecting to agent (192.168.178.28:4444)... [?] Error occurred during installation of tools: [?] Trying to continue anyway...

Environment

Needle Version

Workstation Operating System

Recent Kali Rolling (2018.4)

Python Version

2.7.15+

Python Packages (pip freeze)

adns-python==1.2.1 alembic==1.0.0.dev0 altgraph==0.15 argcomplete==1.8.1 argh==0.26.2 asn1crypto==0.24.0 atomicwrites==1.1.5 attrs==18.2.0 autobahn==17.10.1 Automat==0.6.0 Babel==2.6.0 backdoor-factory==0.0.0 backports-abc==0.5 backports.functools-lru-cache==1.5 backports.shutil-get-terminal-size==1.0.0 BBQSQL==1.0 bcrypt==3.1.4 bdfproxy==0.0.0 beautifulsoup4==4.6.0 biplist==1.0.3 BlindElephant==1.0 blinker==1.4 capstone==3.0.5 cbor==1.0.0 certifi==2018.1.18 cffi==1.11.5 chardet==3.0.4 cheroot==6.2.0 CherryPy==14.0.1 CherryTree==0.38.5 // FIXME: could not find svn URL in dependency_links for this package: chirp===daily-20180815 click==6.7 colorama==0.3.7 ConfigArgParse==0.13.0 configobj==5.0.6 configparser==3.5.0b2 constantly==15.1.0 construct==2.8.16 cryptography==2.2.1 cycler==0.10.0 Cython==0.28.4 decorator==4.3.0 deprecation==2.0.6 dhcpig==0.0.0 dicttoxml==1.7.4 dis3==0.1.2 distorm3==3.3.8 dnslib==0.9.7 dnspython==1.15.0 docutils==0.14 dropbox==8.7.1 easygui==0.96 EasyProcess==0.2.3 EditorConfig==0.12.1 Elixir==0.7.1 entrypoints==0.2.3.post3 enum34==1.1.6 et-xmlfile==1.0.1 feedparser==5.2.1 filedepot==0.5.2 filteralchemy==0.1.0 Flask==1.0.2 Flask-BabelEx==0.9.3 Flask-Classful==0.14.1 Flask-Login==0.4.1 Flask-Mail==0.9.1 Flask-Principal==0.4.0 Flask-Security==3.0.0 Flask-Session==0.3.1 Flask-SQLAlchemy==2.1 Flask-WTF==0.14.2 frida==12.2.18 frida-tools==1.2.2 funcsigs==1.0.2 fuse-python==0.3.1 future==0.15.2 futures==3.2.0 fuzzywuzzy==0.16.0 GDAL==2.3.2 GeoIP==1.3.2 gevent==1.3.7 gitdb2==2.0.5 GitPython==2.1.11 greenlet==0.4.15 h2==3.0.1 hpack==3.0.0 html2text==2018.1.9 html5lib==1.0.1 httplib2==0.11.3 hyperframe==5.1.0 hyperlink==17.3.1 idna==2.6 impacket==0.9.17 incremental==16.10.1 ipaddress==1.0.19 iptools==0.6.1 IPy==0.83 ipython==5.8.0 ipython-genutils==0.2.0 itsdangerous==0.24 javalang==0.11.0 jdcal==1.0 Jinja2==2.10 jsbeautifier==1.6.4 jsmin==2.2.2 jsonpickle==0.9.5 jsonrpclib==0.1.7 keyring==16.1.1 keyrings.alt==3.1 killerbee==1.0 kiwisolver==1.0.1 ldap3==2.4.1 lxml==4.2.5 lz4==1.1.0 M2Crypto==0.31.0 macholib==1.9 Mako==1.0.7 MarkupSafe==1.0 marshmallow==3.0.0b3 marshmallow-sqlalchemy==0.14.1 matplotlib==2.2.2 mechanize==0.2.5 mercurial==4.7.2 metaconfig==0.1.4a1 mockito==0.5.2 more-itertools==4.1.0 msgpack==0.5.6 mysqlclient==1.3.10 nassl==0.12 netaddr==0.7.19 netifaces==0.10.4 NfSpy==1.0 nplusone==1.0.0 numpy==1.14.5 olefile==0.46 openpyxl==2.4.9 OWSLib==0.17.0 packaging==18.0 PAM==0.4.2 paramiko==2.4.0 parse==1.6.6 passlib==1.7.1 pathlib2==2.3.3 pathtools==0.1.2 pcapy==0.10.8 peepdf==0.4.2 pefile==2018.8.8 pexpect==4.6.0 phonenumbers==8.9.10 pickleshare==0.7.5 Pillow==5.3.0 pipenv==2018.10.13 pluggy==0.8.0 pluginbase==0.5 portend==2.2 prettytable==0.7.2 prompt-toolkit==1.0.15 protobuf==3.6.1 psycopg2==2.7.5 py==1.7.0 py-ubjson==0.12.0 pyasn1==0.4.2 pyasn1-modules==0.2.1 pycairo==1.16.2 pycparser==2.18 pycrypto==2.6.1 pycurl==7.43.0.1 PyDispatcher==2.0.5 pydns==2.3.6 pydot==1.2.4 pyenchant==2.0.0 Pygments==2.2.0 PyGObject==3.30.2 pygtkspellcheck==4.0.5 pyinotify==0.9.6 PyInstaller==3.3.1 pylibemu==0.3.3 pyminifier==2.1 pymongo==3.7.1 pymssql==2.1.3 PyNaCl==1.3.0 pyOpenSSL==17.5.0 pyparsing==2.1.10 PyPDF2==1.26.0 pyperclip==1.6.4 pyproj==1.9.5.1 pyrit==0.5.1 pyscard==1.9.7 pyserial==3.4 PySocks==1.6.8 pyspatialite==3.0.1 pysqlite==2.7.0 pytesseract==0.2.0 pytest==3.6.4 python-dateutil==2.6.1 python-editor==1.0.3 python-Levenshtein==0.12.0 python-libnmap==0.7.0 python-magic==0.4.16 python-nmap==0.6.1 python-slugify==1.2.5 python-snappy==0.5.3 pythonaes==1.0 PyTrie==0.2 pytz==2018.4 pyusb==1.0.2 PyV8==1.0.dev0 PyVirtualDisplay==0.2.1 PyX==0.12.1 pyxdg==0.25 PyYAML==3.13 qark==2.0 qrcode==6.0 qt4reactor==1.6 rdpy==1.3.2 readline==6.2.4.1 requesocks==0.10.8 requests==2.18.4 rfidiot==1.0 roman==2.0.0 rsa==3.4.2 scandir==1.9.0 scapy==2.4.0 SecretStorage==2.3.1 selenium==3.5.0 service-identity==16.0.0 Shapely==1.6.4 simplegeneric==0.8.1 simplejson==3.15.0 singledispatch==3.4.0.3 sipvicious==0.2.8 six==1.11.0 slowaes==0.1a1 smmap2==2.0.5 speaklater==1.3 SQLAlchemy==1.2.8 sqlalchemy-schemadisplay==1.3 sshtunnel==0.1.4 stem==1.7.0 subprocess32==3.5.3 tcpwatch==1.3.1 tempora==1.11 tornado==5.1.1 tqdm==4.23.4 traitlets==4.3.2 trollius==2.0.1 Twisted==18.7.0 txaio==2.10.0 typing==3.6.6 u-msgpack-python==2.1 unicodecsv==0.14.1 Unidecode==1.0.22 urllib3==1.22 urwid==2.0.1 uTidylib==0.5 vinetto==0.7b0 virtualenv==16.0.0 virtualenv-clone==0.4.0 volatility==2.6 watchdog==0.8.3 wcwidth==0.1.7 webargs==4.0.0 webencodings==0.5 websocket-client==0.53.0 webunit==1.3.10 Werkzeug==0.14.1 wfuzz==2.3.1 Whoosh==2.7.4 wifite==2.2.5 wsaccel==0.6.2 WTForms==2.2.1 wxPython==3.0.2.0 wxPython-common==3.0.2.0 xlrd==1.1.0 XlsxWriter==0.9.6 xlutils==2.0.0 xlwt==1.3.0 xmlbuilder==1.0 yara-python==3.8.0 zenmap==7.70 zim==0.68 zlib-wrapper==0.1.3 zope.interface==4.3.2

Device iOS Version

10.2.1

marco-lancini commented 5 years ago

Hi! From the snippet above I can see:

cannot copy extracted data for './usr/local/bin/perl' to '/usr/local/bin/perl.dpkg-new': failed to write (No space left on device)

I'm afraid the /system partition of the device run out of space...

HanseSecure commented 5 years ago

Info in the system settings shows me:

space 12,13 GB available 8,62 GB

Update

[on iphone] cp /usr/local/bin/perl /usr/local/bin/perl.dpkg-new cp: cannot stat `/usr/local/bin/perl': No such file or directory

also which didnt find it. i think the error message is missleading. perl isnt installed at this stage

HanseSecure commented 5 years ago

update

Okay i switched to another IPhone 5 iOS 10.3.3 and noticed the following:

solved 'issues'

  1. Frida Repos: worked on that IPhone, so it's not an issue of needle ;-)

  2. Space Moved several folders from the system partition to the private one (thanks symlinks ;-) and the above error warning (space) vanished

new issues

package org.coolstar.iostoolchain

[D] Adding repo to cydia.list: http://coolstar.org/publicrepo/ [D] [REMOTE CMD] Remote Command: echo "deb http://coolstar.org/publicrepo/ ./" >> /etc/apt/sources.list.d/cydia.list [D] [REMOTE CMD] Remote Command: apt-get update [D] [REMOTE CMD] Remote Command: apt-get install -y --force-yes org.coolstar.iostoolchain [!] E: Broken packages

fsmon

[V] [INSTALL] Manually installing: FSMON [*] Pushing: /home/hansemann/Software/needle/needle/libs/devicetools/fsmon -> /usr/bin/fsmon [D] Uploading: "/home/hansemann/Software/needle/needle/libs/devicetools/fsmon" -> "/usr/bin/fsmon" [D] [LOCAL CMD] Local Command: sshpass -p "alpine-2018!" scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -P 2222 -r "/home/hansemann/Software/needle/needle/libs/devicetools/fsmon" root@127.0.0.1:"/usr/bin/fsmon" [D] [REMOTE CMD] Remote Command: chmod +x /usr/bin/fsmon [!] chmod: cannot access `/usr/bin/fsmon': No such file or directory

clutch

V] [INSTALL] Manually installing: CLUTCH [D] [REMOTE CMD] Remote Command: curl -ksL "http://cydia.iphonecake.com/Clutch2.0.4.deb" -o /var/root/clutch.deb [D] [REMOTE CMD] Remote Command: dpkg -i /var/root/clutch.deb && rm -f /var/root/clutch.deb [D] [REMOTE CMD] Remote Command: killall -HUP SpringBoard [D] [REMOTE CMD] Remote Command: which pbwatcher [V] [INSTALL] Manually installing: PBWATCHER [*] Pushing: /home/hansemann/Software/needle/needle/libs/devicetools/pbwatcher -> /usr/bin/pbwatcher [D] Uploading: "/home/hansemann/Software/needle/needle/libs/devicetools/pbwatcher" -> "/usr/bin/pbwatcher" [D] [LOCAL CMD] Local Command: sshpass -p "alpine-2018!" scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -P 2222 -r "/home/hansemann/Software/needle/needle/libs/devicetools/pbwatcher" root@127.0.0.1:"/usr/bin/pbwatcher" [D] [REMOTE CMD] Remote Command: chmod +x /usr/bin/pbwatcher [!] chmod: cannot access `/usr/bin/pbwatcher': No such file or directory

additonal info

i noticed that the org.coolstar.scp-sftp-dropbear package of the dependency_installer seems to install a 64bit version of scp/ssh which also leads to trouble. Fixed this through install native openssh from the cydia store

Could you please tell me some modules which use the above packages (failed to install) ? Modules in binary/info just work fine.

Best Regards

btw you really do a amazing job with you projects ;-) https://twitter.com/HanseSecure/status/1069842872395022338

marco-lancini commented 5 years ago

Hey, thanks for the updated info. @Yogehi, can you have a look please?

Yogehi commented 5 years ago

I'll look at it tomorrow. Was too busy today :( should be easy to remediate

Yogehi commented 5 years ago

some updates.

org.coolstar.iostoolchain i wasn't able to replicate the issue, but i want to try something. i made the following changes to the device/dependency_installer module:

i'm HOPING something with the above fixed your issue @HanseSecure.

fsmon not able to reproduce :( i even set my password to the same alpine-2018! value:

[needle][dependency_installer] > run
[*] Checking connection with device...
[+] Already connected to: 127.0.0.1
[D] Creating temp folder: /var/root/needle/
[D] [REMOTE CMD] Remote Command: if [ -d /var/root/needle/ ]; then echo "yes"; else echo "no" ; fi
[*] Checking prerequisites...
[D] [REMOTE CMD] Remote Command: which apt-get
[D] [REMOTE CMD] Remote Command: which dpkg
[D] [INSTALL] Already installed: coreutils.
[D] [INSTALL] Already installed: coreutils-bin.
[*] Refreshing package list...
[D] [REMOTE CMD] Remote Command: dpkg --get-selections | grep -v "deinstall" | cut -f1
[D] [REMOTE CMD] Remote Command: cat /etc/apt/sources.list.d/cydia.list
[*] The following tools are going to be installed: ['FSMON']
[>][QUESTION] Do you want to continue? [y/N]: y
[D] [REMOTE CMD] Remote Command: which fsmon
[V] [INSTALL] Manually installing: FSMON
[*] Pushing: /root/Programs/needle/needle/libs/devicetools/fsmon -> /usr/bin/fsmon
[D] Uploading: "/root/Programs/needle/needle/libs/devicetools/fsmon" -> "/usr/bin/fsmon"
[D] [LOCAL CMD] Local Command: sshpass -p "alpine-2018!" scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -P 2222 -r "/root/Programs/needle/needle/libs/devicetools/fsmon" root@127.0.0.1:"/usr/bin/fsmon"
[D] [REMOTE CMD] Remote Command: chmod +x /usr/bin/fsmon

@HanseSecure would you be able to do the following for me:

i'll look into the other issues later this week. for now, here are the git changes in the develop branch if you want to try the new stuff:

https://github.com/mwrlabs/needle/commits/develop/needle/core/device/device.py

https://github.com/mwrlabs/needle/commits/develop/needle/core/utils/constants.py

https://github.com/mwrlabs/needle/blob/develop/needle/modules/device/dependency_installer.py

HanseSecure commented 5 years ago

Hey @yogehi

Thanks for the help. Will test the dev git && report your suggestion next week.

Regards Flo

HanseSecure commented 5 years ago

Hey @Yogehi

Thanks for your help. Decided to try it on a fresh new Kali and it works.

Thanks

Regards Flo