Signed App Installation

[PythEch]

So apparently there was a python implementation of libimobiledevice here: pymobiledevice

It is written in Python meaning that you won't even need to compile it as that is an interpreted langauge. Downside is, we need to use PyInstaller for Windows distribution because Python is not included in Windows, unlike Mac and Linux.

Anyways, if you don't want cross-language development (in other words, if you want just pure Java), we can look at that code and use it as reference to figure out how mobile installation_proxy is used. Since Python is a high-level language, it'll be easier to understand how things can be ported to Java.

I did those on a Windows machine, though it is relatively easier to get this done on Unix.

Requirements: Python 2.7 M2Crypto Construct

After the installation, head over to pymobiledevice and download the zip package. Extract pymobiledevice-master to C:\Python27 rename it to something like pymobiledevice otherwise Python will give syntax error. After that, fire up IDLE (Python GUI) from Start.

Now welcome to the Python Shell. You'd be surprised because all we need is 3 lines of code:

>>> from pymobiledevice import apps
>>> lockdown = apps.LockdownClient()
Connecting to device: 0aa4cc**********************************
Found pairing record for device 0aa4cc**********************************
>>> apps.mobile_install(lockdown, "C:\\Python27\\app.ipa")
Connecting to device: 0aa4cc**********************************
Connecting to device: 0aa4cc**********************************
Installing, C:\Python27\app.ipa: 5 % Complete
Installing, C:\Python27\app.ipa: 15 % Complete
Installing, C:\Python27\app.ipa: 20 % Complete
Installing, C:\Python27\app.ipa: 20 % Complete
Installing, C:\Python27\app.ipa: 30 % Complete
Installing, C:\Python27\app.ipa: 40 % Complete
Installing, C:\Python27\app.ipa: 50 % Complete
Installing, C:\Python27\app.ipa: 60 % Complete
Installing, C:\Python27\app.ipa: 70 % Complete
Installing, C:\Python27\app.ipa: 80 % Complete
Installing, C:\Python27\app.ipa: 90 % Complete
Installation Complete

Or you can just:

C:\Python27\pymobiledevice>apps.py --install=C:\Python27\app.ipa
Connecting to device: 0aa4cc**********************************
Found pairing record for device 0aa4cc**********************************
Connecting to device: 0aa4cc**********************************
Connecting to device: 0aa4cc**********************************
Installing, C:\Python27\app.ipa: 5 % Complete
Installing, C:\Python27\app.ipa: 15 % Complete
Installing, C:\Python27\app.ipa: 20 % Complete
Installing, C:\Python27\app.ipa: 20 % Complete
Installing, C:\Python27\app.ipa: 30 % Complete
Installing, C:\Python27\app.ipa: 40 % Complete
Installing, C:\Python27\app.ipa: 50 % Complete
Installing, C:\Python27\app.ipa: 60 % Complete
Installing, C:\Python27\app.ipa: 70 % Complete
Installing, C:\Python27\app.ipa: 80 % Complete
Installing, C:\Python27\app.ipa: 90 % Complete
Installation Complete

[alexhulbert]

I tried it myself, and was pleasantly surprised at the error I got:

socket.sslerror: [Errno -1] SSL protocol exception: Differences between the SSL
socket behaviour of cpython vs. jython are explained on the wiki:
http://wiki.python.org/jython/NewSocketModule#SSL_Support

This is looking great!

[alexhulbert]

Aha! We need to use Jython's native SSL/Socket module (I believe its called "newsocketmodule"). It could be as easy as just changing the imports, but some parts may go more in depth (I hope not). I'll see what I can do and let you know what happens.

EDIT: This may be of significance: http://tech.pedersen-live.com/2010/10/trusting-all-certificates-in-jython/ One major difference is that unlike cpython, Jython verifies all certificates with Java. This can be turned off, luckily.

[PythEch]

The problem is not the verification :( (used trust all certificates trick but didn't work) You can see that if you replace raise _map_exception(jlx) with raise in socket.ssl.__init__(). Map exception actually gives a misleading error I think. When you use raise, the actual error says Issuer cannot be empty, but it is not. Jython is using old socket.ssl() (which is deprecated) instead of ssl.wrap_socket(). Not only that, when I looked at the code:

# Define the SSL support

class ssl:

    def __init__(self, plain_sock, keyfile=None, certfile=None):
        try:
            self.ssl_sock = self._make_ssl_socket(plain_sock)
            self._in_buf = java.io.BufferedInputStream(self.ssl_sock.getInputStream())
            self._out_buf = java.io.BufferedOutputStream(self.ssl_sock.getOutputStream())
        except java.lang.Exception, jlx:
            raise #notice I actually changed here

    def _make_ssl_socket(self, plain_socket, auto_close=0):
        java_net_socket = plain_socket._get_jsocket()
        assert isinstance(java_net_socket, java.net.Socket)
        host = java_net_socket.getInetAddress().getHostAddress()
        port = java_net_socket.getPort()
        factory = javax.net.ssl.SSLSocketFactory.getDefault();
        ssl_socket = factory.createSocket(java_net_socket, host, port, auto_close)
        ssl_socket.setEnabledCipherSuites(ssl_socket.getSupportedCipherSuites())
        ssl_socket.startHandshake() # throws exception here
        return ssl_socket
[...]

I noticed that keyfile and certfile parameters are not even used. Then how it understands where the certificate is? CPython equivalent calls an OpenSSL based C function. I believe you'll have more luck with this as this actually is a Java related problem. I think it wants certificates from socket somehow. I've no exprience regarding SSLSocket on Java

[alexhulbert]

Should I try grabbing the bleeding-edge ssl.py from the source code? That might work. I'm going to replace the 2.7b1 one with the latest.

[PythEch]

Sadly, it's just this:

import socket

wrap_socket = socket.ssl

Maybe Java doesn't like iPhone's response so that's why it gives those errors.

[alexhulbert]

I checked too. You can see the source at https://bitbucket.org/jython/jython/src/b4ae9e2893c8f8748bc44595f04559d586f96754/Lib/socket.py?at=default#cl-1820. Its the same deal, both parameters aren't used

[PythEch]

Yes, actually even it's a better version of 2.5.3's socket.py, still gievs the same error.

[alexhulbert]

What should we do from here? If we handled usbmux.py's sockets with BouncyCastle, we could go back to the old non-itunes-dependant version of valitate_pair/pair. But where would we start? I know python, but I'm pretty much clueless when it comes to SSL and Sockets. We may need some more people, but this is a sort of specialized field (you'd have to know about X509, SSL, Sockets, Java, and Python) Any idea where we could find somebody?

[PythEch]

The non-itunes-dependant version of validate_pair doesn't work, I tried that with original pymobiledevice's pair_records. Actually Jython doesn't even load the certificates I think. Even Google doesn't know that goddamn error. We have a problem with SSL Socket, not usbmux.

For the other questions, I fully agree with you. I have the same feelings, and no, I really don't know that type of person :( Maybe we can ask a question on StackOverflow and hope Jon Skeet will answer our question, haha

If I find a solution I'll let you know.

[alexhulbert]

Since this is "Jython," we still have the ability to reference Java from within the python code. My thought is that instead of relying on Jython/Python's SSL, Socket, and Cryptography libraries, we reference Java. We could use BouncyCastle for ca.py and the SSL stuff. Then, we could use jnr-unixsocket for UNIX sockets and Java's built in Socket factory for INET sockets. Now all we need is somebody who can do that :).

EDIT: come to think of it, that would also make this linux-compatible! I think this might be the optimal way to go from here.

[PythEch]

If only that was as easy as it sounds :)

EDIT: I agree, that's the only way, I guess. Seems like Jython's socket library is cumbersome and outdated

[alexhulbert]

I'm going to try posting this up on /r/jailbreakdevelopers, but where can I get help with sockets and SSL? There must be some kind of irc server where cryptography people hang out. I'll try to find one.

[alexhulbert]

I have awesome news!!!! I asked on a cryptography channel for irc. Evidently Jim Baker (the guy who helped with the first issue) made a newer implementation of sockets! You can see it here

[alexhulbert]

Damnit! I just looked [https://github.com/jimbaker/socket-reboot/blob/master/_socket.py#L384] only to find that he hasn't reimplemented those two parameters... I guess we're going to have to find someone after all.

[PythEch]

I gave it a try but couldn't manage to make it work (errors, error nightmare!! : https://ghostbin.com/paste/oadf9)

Still can't believe installing applications became a lot of trouble. Maybe we can go back to libimobiledevice-wrapper and finish the missing pieces? What do you think?

[PythEch]

It seems like I need to get educated about this. Python 2.5.3 gives errror socket.sslerror: (8, 'EOF occurred in violation of protocol') when using socket.ssl() method. This doesn't look like Jython's fault. Now we need complete ssl.wrap_socket alternative.

Also about ignored keyfile, certfile parameters. Jython doc says:

Java manages certificates completely differently to cpython: see socket module documentation on SSL for more information. For this reason, all parameters other than the first sock parameter are ignored.

I think we need BouncyCastle

[alexhulbert]

I think netty should be perfect for sockets. A friend of mine is just starting to use it and from what he's said, its not too-too hard (but none of use know anything about sockets). It might be easier to just use Java's built-in sockets. Whatever happens, I think a great place to start would be send and recv in usbmux.py. Those seem to be the core of the regular socket stuff. I'm going to familiarize myself with plist_service.py and try to figure out what's going on. The SSL stuff in there is going to be tricky. When (or if) we finish that, we can work on recreating ca.py using bouncycastle. That should be easier than it sounds, as bouncycastle has built-in functions for automatically creating X509 certificates and Certificate Authorities.

EDIT: Looks like the only function we need to work on in plist_service.py is "ssl_start." Everything else boils down to two core functions: usbmux.SafeStreamSocket.recv and usbmux.SafeStreamSocket.send. This is going to be easier than I thought!

EDIT 2: Its pretty simple to create sockets in java! Look at this example file.

EDIT 3: How on earth are we going to implemet wrap_socket? I don't know if or how Java can do this.

[alexhulbert]

Looks like a lot of people want to beta test! I'm going to make a shell script that runs some misc. commands like AFCShell.do_ls('') and apps.mobile_install() and have the user say if they got any errors.

[PythEch]

Looks like I will have a lot of free time in two weeks. Most probably I'm going to be very busy these days, just wanted to let you know.

About your questions, I don't have much experience on Java, but it seems like it's because of the difference between SSL and TLS protocols.

[alexhulbert]

While your busy doing whatever you're doing, I'll work on porting the UNIX Socket part myself. JNR-Unixsocket is really simple, so that shouldn't be too hard. As for the INET sockets and SSL stuff, people are continuing to see the post on /r/jailbreakdevelopers, so somebody who's good at networking is bound to appear sometime soon. The only down side is that not a lot of people know Java anymore.

[alexhulbert]

Sorry I haven't been helping recently, but I should be able to contribute more now :). I'll finish up the sockets later today. Good find with the socket thing!

[PythEch]

not-yet-commons-ssl actually works (had no time to explain better). There is a problem with it though, socket connection gets rejected after the first data recieved.

https://ghostbin.com/paste/hkxee

If you managed to get unix sockets work this issue will be solved. Now I'm looking at my comments, sorry for complaining over and over again

[alexhulbert]

I said that because I was complaining... I guess that makes us even :P. My computer just stopped working a couple hours ago. The screens all glitchy and it barely boots up. Luckily, I have square trade, so I'm going to send it in and it should be back. I backed up my hard drive and I should be able to recover everything. I'm posting this through an old clunky computer of mine. I just couldn't wait to test this thing out :). Did you make sure that it makes an INET socket for Windows and a UNIX one for mac/linux? I'll get Python/Jython and see if I can get everything (jython itself and the program) running smoothly.

[PythEch]

Oh that's sad, hope it's not critical and hope you repair it soon :(

It just creates an INET socket because stock Jython socket.py does not support UNIX sockets. I just modified some things in socket.py to make it work with commons-ssl. For some reason, socket sends random data when calling send() for the second time, and then iPhone rejects connection.

[alexhulbert]

Weird... I'm currently wiping the old computer so I can try out that ssl library. Once thats done, I'll try manually sending and receiving with the socket using SocketTest. My laptop should be there and back by next Friday, but I can still try everything like this.

[PythEch]

I was hiding my UDID but I don't care anymore, I'm too lazy right now to edit. Here is what I get: https://ghostbin.com/paste/k8ddk

Line 156-162 makes trouble. And for whatever reason it omits the first byte (in line 164), should be 00 00 01 1E 3C 3F 78 6D instead of 00 01 1E 3C 3F 78 6D 6C

Also, great find, that looks like a great utility

[alexhulbert]

This may seem like a stupid fix, but how about duplicating the first byte in that request? That might work (I think). I'm still waiting for my computer. They sent it back because "the return sheet had a misprint," whatever the hell that means. Anyway, I'm going to have to send it back in again...

[PythEch]

Not a stupid fix really, but I tried it that day, no luck :( There may be something wrong in the configuration of communication, since commons-ssl is poorly documented. I tried looking at the source code but I noticed nothing wrong with it.

[alexhulbert]

That's what I figured. I'll attempt to run Jython on my old computer again today. It didn't work last time, but I can try a couple other things.

[PythEch]

Forgot to say that you have to use cpython pymobiledevice's pair records to make it work

[alexhulbert]

What do you mean? Can you still use the ones from iTunes? I got Jython working! I'm downloading the repo to my computer now.

[PythEch]

Ah sorry, I mean you have to point home folder to

HOMEFOLDER = os.path.join(os.environ["HOMEPATH"], ".pymobiledevice")

so it will use ca.py generated records. iTunes certificates have no issuer (java will give its goddamn error), I don't know if it can be fixed. I'll try to make it work with the ones from iTunes if I can resolve 2nd-time-write issue first.

[alexhulbert]

I guess thats kind of good in the sense that it makes us implement linux compatibility off the bat.

(I'm going to abbreviate not-yet-commons ssl as nycs. Its a huge pain in the ass to type :) ) I think I have an idea on how do do this. It looks like nycs supports the creation of x509 certificates, which is what ca.py did. Where did you find the Jython wrapper for not-yet-commons ssl (socket.py)? If "socket.py" comes with other files, there might also be a wrapper for nycs's Cryptography functions. If not, I think I might be able to salvage at least part of ca.py or use some of the output from pair() or validate_pair() to add the issuer to the request. After all, all the data we need, including the issuer, is still being send when those functions are called.

EDIT: Sorry if this is a stupid question, but are the two errors we're having related? Does the issuer problem have anything to do with why the first byte is missing?

[PythEch]

I didn't find a wrapper, I wrote it from my mind from the Client example. And yeah I agree about ca.py, that's what I'm thinking, it's the best part of nycs, we can use it for that job. Though I found this today:

http://stackoverflow.com/questions/1615871/creating-an-x509-certificate-in-java-without-bouncycastle

About empty issuer dn, I am thinking of something like this:

import sun.security.x509.CertificateIssuerName

ORIGINAL_GET = sun.security.x509.CertificateIssuerName.get

def jython_monkey_patch_get(name):
    if name.lower() == dn_name:
        print "jython_monkey_patch_get: BANANAS!"
        return "I AM THE HAPPIEST ISSUER MONKEY EVER"
    else:
        print "jython_monkey_patch_get: Original ..."
        return ORIGINAL_GET(name)

sun.security.x509.CertificateIssuerName.get = jython_monkey_patch_get

Response to EDIT: I really don't know, I'll have more time to test soon, right now I am busy. I guess I'll try to do both one-by-one. I didn't get any SSL related lessons so far :D Everything that I know about SSL comes from the internet, btw I really like what this article says: http://softarc.blogspot.com.tr/2011/03/2-way-ssl-with-java-keystore-strikes.html

You'd think with how prevalent SSL/TLS on the web that SSL with Java is easy, well it's not. In addition to having to get to grips with understanding how private/public key, certificates work there's all the networking that goes on in Java.

If you start off trying to get to grips with the in-built Java Secure Sockets Extension you're gonna be stunned by the complexity of it all.

Clear as mud eh? The JSSE guide alone is 73 pages long - but I hate to say that its REALLY a must read. Print it out (double-sided to save some trees) grab a Venti decaff or two and wrestle the information into your brain.

[alexhulbert]

Hi! I just saw your comment. I just wanted t complement you on socket.py. That's what I call some kick-ass code right there. 10 issuer bananas out of 10 :) Anyway, I'm going to test out and debug the byte issues in socket.py by setting up a controlled socket environment. I'll let you know what happens.

[PythEch]

In fact that "kick-ass" monkey patch doesn't work in java runtime :D This would be a better way to do the previous thing, but this doesn't work either:

from sun.security.x509.X500Name import isEmpty

isEmpty = lambda self: False

I hope you'll find out what's wrong so we can continue, good luck!

[alexhulbert]

I was referring to socket.py as kick-ass. Anyway, rather than use iTunes' certificates, I think it would be better to make our own. That way you don't have to pair it with iTunes first and it works on Linux.

[PythEch]

Lol I clearly misread it, that's hilarious (part of the reason is my broken English).

Yeah, turns out ca.py is a must. Sorry about flooding the github page, I am trying a few things at home but they don't work, when they work I either push a commit or comment here for a future reference.

EDIT: How rude, I forgot to say thank you for your compliments :( In fact the only thing that is done by me was openssl_socket function (which is a weird name choice). I guess you didn't open Jython 2.7b1's socket.py before (I have to admit, the new one is better) and that looked somewhat different to you. I don't want to steal credit

[PythEch]

From http://www.reddit.com/r/jailbreak/comments/1w43k7/reminder_install_afc2add_now/ceyodn3:

I presume/hope the package correctly uninstalls when you remove it; to be clear, the package is not inherently dangerous afaik: it simply adds functionality to access the root of your filesystem without a password while connected over USB, which is generally "a bad idea". I think it is safer on iOS 7 (due to "trust this computer" prompts), but I have been told by "people I really trust on this sort of thing" that the prompt is easily bypassed and the reason my code was getting the prompt is because I was "playing by the rules" in a way that wasn't really required. I will say that when pod2g first started asking after this he was quite bothered that it had been installed by default for so long, and wanted to prevent it from being installed in the future. Again: it isn't necessary as OpenSSH (with a password) is better and people using AFC2 to fix things tend to cause more problems by accident due to its limited model of the filesystem.

So according to saurik, it is possible to use afc without trust this computer dialogue. Interesting...

[alexhulbert]

I think that would be the ideal course of action, but there's the obvious question: "how?" There's got to be a page on it somewhere... If worse comes to worse, I guess I could contact Saurik or pod2g. I'm replying to the comment now.

My computer still hasn't come in yet. Its been weeks, and they just keep sending it back to me for different reasons. I hope square trade can finally return me a working computer this time :). You'll see me start to help a lot more once I've gotten it back.

[PythEch]

Well, it takes at least a month to make a computer fixed in here. I even paid 200$ for faulty design of Lenevo, and it had so called 2 years guarantee, good luck

[alexhulbert]

I got my computer back :). I'm downloading the repo now. I'll try running it and I can debug any errors from there.

[alexhulbert]

I got to the point where I got the empty issuer DN error. I don't think we need a specific issuer DN. I believe its just a way of identifying the issuer. I'm pretty sure we can set it to whatever we want. The only problem is: how. The certificates already been created by iTunes, so we can't edit it. The good news is I think I've come up with a solution. We can "cheat." I was thinking we could modify the source code of not-yet-commons ssl so that when it parses the certificate, it automatically adds our issuer DN instead of raising an exception. I'm going to see if this is possible.

EDIT: Looks like the certificates are parsed by Java, not not-yet-commons-ssl. I'm going to try rebuilding ca.py using java.security.cert.X509Certificate

[alexhulbert]

I've changed my mind again. After researching a little bit on the trust dialog, I found this: https://github.com/libimobiledevice/libimobiledevice/issues/20

I think this is what Saurik was talking about. It also includes that issue I was having with the iPad. This seems to be the stablest method. I just need more information

[PythEch]

Great, hope it's in good situation

You can solve the empty issuer DN error by using ca.py generated self-signed certificates, it just works. I'll try to fix that error if I can get ssl working

About last comment, actually although I may missed a point, I don't think that's what Saurik is talking about. They only say it is possible to get UDID of the device without Trust This Device.

Also it is relatively easy to fix validation part to fit it to new iOS 7 style, but I don't think it is critical because you can solve that problem if you temporarily:

• Run apps.list_apps(lockdown)
• Tap Trust on the device
• Re-run the command, it'll give an error
• iTunes auto-launchs
• Re-run the comamnd for the last time, you have a working pair record

I can't fix the second time write issue for weeks, and I don't know the cause, sorry

[alexhulbert]

Sorry, I haven't been doing much in relation to our port of pymobiledevice, but I think I've found something that may make our lives a hell of a lot easier. As of a couple days ago, jimbaker/socket-reboot has sufficiant support for certificates in wrap_socket. I haven't tested it, so it might not be complete. If it isn't, it probably will be within a couple days. I'm going to try using Jimbaker's version of SSL support. I'll let you know what happens.

[PythEch]

I wasn't commenting like you but that doesn't mean I didn't do anything with it. I built libimobiledevice with Cygwin and was trying to merge some of the parts of libmobiledevice using libimobiledevice-wrapper with pymobiledevice through JNA. As you can guess, that's not the best workaround and it became a mess. C/C++ Cygwin -> JNA -> Java -> Python -> Jython -> Java

Also I wasn't successful at understanding what the limd-sdk was supposed to do.

About latest socket-reboot, at the moment Java doesn't trust pymobiledevice generated certificates, trying to make it work.

EDIT: It seems it has the exact same problem. But this time it says Handshake is incomplete, that may be the reason.

https://ghostbin.com/paste/9wd2a and more detailed one with -Djavax.net.debug=all

[alexhulbert]

I tried using JNA with a cygwin-compiled version. I completely and totally failed. I even made the whole programming language diagram that you did when discussing it on irc :). I'll post some psudeocode for how I think the java port of ca.py should go. You can correct me on some stuff that I'm unsure about, and then I should be able to completely port it. Once that's done, we can work on getting that incorporated into usbmux, plist_service, etc.

[PythEch]

Well, it seems we have similar minds, haha

The internet says incomplete handshake means incomplete chain of trust. It says we should provide another CA Cert but tbh, I don't really understand what these really mean and why Python doesn't have to do all these things.

This may be wrong, but what I understand from this diagram (from) is that empty issuer DN is really a problem.

I'm sorry for my lack of understanding. I'll try to do some changes with ca.py, sometimes random experiments work.

Also I'd like to know the IRC channel you are talking about

[alexhulbert]

I don't think its as big of a problem as you think. Since these are self-signed certificates (I don't think "The Issuer Monkey" is a real Certificate Authority), I think you just have to have an issuer DN. It doesn't need to be anything special.

---

I think I've got a pretty good idea of how bouncycastle works. Nodepad++ corrupted my psudeocode :/. I'll get an irc channel setup on my website within a week (I tried a generic chat program, but there really aren't any good ones) and we can collaborate on and discuss the Java port of ca.py. You seem to know more than me about Certificates (I cant get wrap my head around the whole concept), so thats a start.