venomous0x / WhatsAPI

Interface to WhatsApp Messenger
2.59k stars 2.14k forks source link

discussing a way to new Login/Auth/Challenge/KeyStream Mechanism #422

Closed assegaf closed 10 years ago

assegaf commented 10 years ago

hi,

just notice about this function

protected function sendMessageReceived($msg) .... $messageHash["type"] = "chat"; ....

I see this also executed doing many message notification, like this one (receive profile image change) : rx [message from="628151XXXXXX@s.whatsapp.net" id="1172606828" type="notification" t="1374144652" ] rx [notify xmlns="urn:xmpp:whatsapp" name="Nadia Makes"][/notify] rx [request xmlns="urn:xmpp:receipts"][/request] rx [notification type="picture"] rx [set jid="628151XXXXXX@s.whatsapp.net" id="1374144652"][/set] rx [/notification] rx [/message]

is it that we must return the same type as it, so as "notification" not "chat"

Dynogic commented 10 years ago

Ahhh....

            else if (ProtocolTreeNode.TagEquals(node, "message"))
            {
                var notification = node.GetChild("notification");
                if (notification != null)
                {
                    var picture = notification.GetAttribute("type");

                    if (picture != null && picture == "picture")
                    {
                        UpdateBubbleGroups();
                        return null;
                    }
                }

                var from = node.GetAttribute("from");
                var time = node.GetAttribute("t");

                var typing = node.GetChild("composing");
                if (typing != null)
                {
                    var tb = new TypingBubble(long.Parse(time), Bubble.BubbleDirection.Incoming,
                                              @from, null, false, this, true); 
                    return tb;
                }
                var paused = node.GetChild("paused");
                if (paused != null)
                {
                    var tb = new TypingBubble(long.Parse(time), Bubble.BubbleDirection.Incoming,
                                              @from, null, false, this, false); 
                    return tb;
                }

                if (node.GetChild("received") == null && node.GetAttribute("type") == "chat" 
                    && !from.Contains(WhatsConstants.WhatsAppGroupServer))
                    SendMessageReceived(node);

....

That's the start of my message branch. Typing [and not typing] seem to be weirdly grouped up in message nodes in WhatsApp. You also get profile picture updated notifications in there.

Those are the two exceptions I've found so far.

I took your advise and added

&& node.GetAttribute("type") == "chat" 
                    && !from.Contains(WhatsConstants.WhatsAppGroupServer))

to my SendMessageReceived condition.Therefore, if it isn't type of "chat", it shouldn't send. Moreover, their is no received checkmark in the official WhatsApp client for groups. So I've added that in there as well.

Maybe this fixes this weird blocked issue?

assegaf commented 10 years ago

I catch you saying only message type chat, send back SendMessageReceived. the other just ignore them.

without sniffing real whatapp smartphone what those client do , we dont know for sure. I hope this remove blocked issue. I will try like your suggestion for like 24 message a day. will post here the result.

Dynogic commented 10 years ago

I haven't been banned yet ;)

Once my second ban happens, I'll throw the towel in and start the sniffer up.

But, you can always do it yourself! https://github.com/shirioko/MissVenom

^^ You just modify your router's DNS and MissVenom acts as a reverse proxy.

assegaf commented 10 years ago

not good, if we dont SendMessageReceived , any other tx send will fail, I already tested it, when relogin, the notification are resend again all of them by whatsapp server.

assegaf commented 10 years ago

oh .. just noticed about missVenom, will try this weekend to find out exact behavior of android client.

Dynogic commented 10 years ago

"not good, if we dont SendMessageReceived , any other tx send will fail, I already tested it, when relogin, the notification and resend again all of them by whatsapp server."

^^ You have to rephrase that. I don't understand. >.<

assegaf commented 10 years ago

here is the exact what happen, we skipped the SendMessageReceived right except for chat right .

then notification of profile image change occurs, eg. rx [message from="628151XXXXXX@s.whatsapp.net" id="1172606828" type="notification" t="1374144652" ] rx [notify xmlns="urn:xmpp:whatsapp" name="Nadia Makes"][/notify] rx [request xmlns="urn:xmpp:receipts"][/request] rx [notification type="picture"] rx [set jid="628151XXXXXX@s.whatsapp.net" id="1374144652"][/set] rx [/notification] rx [/message]

we skip SendMessageReceived() for this.

but when

try to do tx , example sendActiveStatus() failed, to write, got exception,

try to relogin. notification occurs again :

rx [message from="628151XXXXXX@s.whatsapp.net" id="1172606828" type="notification" t="1374144652" ] rx [notify xmlns="urn:xmpp:whatsapp" name="Nadia Makes"][/notify] rx [request xmlns="urn:xmpp:receipts"][/request] rx [notification type="picture"] rx [set jid="628151XXXXXX@s.whatsapp.net" id="1374144652"][/set] rx [/notification] rx [/message]

Dynogic commented 10 years ago

...and... second account banned. >.<

shirioko commented 10 years ago

EDIT: this one: http://www.wpcentral.com/whatsapp-windows-phone-gets-another-minor-update-today

assegaf commented 10 years ago

:D ... I wonder what could possible goes wrong. lets try sniff mrvenom tommorow.

I think we use the Android Token (nothing to do with WP8).

Dynogic commented 10 years ago

@shirioko what does that mean? subtle update ... hmm....

@assegaf mind my ignorance above =) looking at https://github.com/tgalal/yowsup/blob/master/src/Yowsup/connectionmanager.py it can be seen the various message receipts sent in parseMessage. Perhaps this is why we're getting blocked... I don't think anyone has been blocked from using Wazzap =)

@shirioko @assegaf could you test Code Request for me on the new token (Android) mentioned on here 2-3 days ago? i'm trying to do a request now... it states it's been sent, but I'm getting no SMS. worked yesterday just fine >.<

shirioko commented 10 years ago

This update popped up on my phone this morning. WhatsAppNative.dll seems to be the same as in the previous version. I'll do a diff when I get home from work. I've already uploaded the new files.

Dynogic commented 10 years ago

@assegaf, could we speak outside of GitHub? Perhaps over Skype text?

assegaf commented 10 years ago

removed whatsup number

Dynogic commented 10 years ago

@ removed skype info.

assegaf commented 10 years ago

ok sent request, remove/edit your post now, :)

shirioko commented 10 years ago

Interdesting... 496-506_diff

Dynogic commented 10 years ago

@shirioko hmmm.. mind sending 506?

shirioko commented 10 years ago

@Dynogic it's on my server

So I only compared WhatsAppCommon.dll which is the functional part rather than WhatsApp.dll, which is the front end part.

What the diffs showed so far:

Comparing WhatsApp.dll right now..

Dynogic commented 10 years ago

@shirioko, link me? :P

shirioko commented 10 years ago

Differences in WhatsApp.dll 2.10.501.0:

So overall not that exciting, no wonder they didn't bother producing a changelog. Just cleaning up from the error logs they received.

@Dynogic http://mywapi.nl/whatsapi/share/2.10.506/Install/WhatsAppCommon.dll

assegaf commented 10 years ago

I closed this issues, cause this fix (which already did) seem not helping blocking issues, current api working just fine with or without fix.

assegaf commented 10 years ago

I see, after long try, sniff with wireshark, I notice something different, whatsapi vs real client android, at first beginning of doLogin() to get challenge string.

whatsapi receiving this : hexcoded [000005f8030141ab00000df802bbf802f8019cf803e4cb0c00001bf8041be8cffc14dda64d5863366f524fabe055bfde7d40c410a5a800]

rx [start from="s.whatsapp.net"][/start] rx [stream:features] rx [receipt_acks][/receipt_acks] rx [w:profile:picture type="all"][/w:profile:picture] rx [/stream:features]

rx [challenge xmlns="urn:ietf:params:xml:ns:xmpp-sasl"]ݦMXc6oRO??U??}@???[/challenge]

so, its looks okay.

I sniffed real clients, [000005f8030141ab00000df802bbf802f8019cf803e4cb0c80004a5abf70e6b9b3024a12209e1753fdeddee01a301a37d0a2c7b26555bab9d7fbe7d73a6ba8c9043e8ad72c9c8dab3d446bd590c57af8264f9b47408fe094a78ac117ab12b45d8720780604]

rx [start from="s.whatsapp.net"][/start] rx [stream:features] rx [receipt_acks][/receipt_acks] rx [w:profile:picture type="all"][/w:profile:picture] rx [/stream:features]

quite different, missing the challenge node (never found challenge node), I could not decode this : 80004a5abf70e6b9b3024a12209e1753fdeddee01a301a37d0a2c7b26555bab9d7fbe7d73a6ba8c9043e8ad72c9c8dab3d446bd590c57af8264f9b47408fe094a78ac117ab12b45d8720780604

neither,

we are not even get challenge string, not sure to decode further message not even make key to further as decoding chiper.

FYI, @davidgfnet

shirioko commented 10 years ago

@assegaf whatsapp uses rc4 key stream encryption, that's why you can't decode it. I've seen this behaviour before, whatsapp can reuse key streams across socket connections. It doesn't need to reauthenticate on each connection like whatsapi does.

Dynogic commented 10 years ago

@shirioko , I think @assegaf was right originally about them detecting bans from ACKs not being sent back correctly. I'm going to experiment more after I finish reimplementing this code request system.

assegaf commented 10 years ago

managed to found ways to decrypt it, used php whatsapi to decrypt it (indeed rc4), main point it is, the whatsapp client seem save too the challenge string, so only when after register they send it.

TX didnot have clue to decrypt it !,

so far this is RX sequence after register. (1). rx [start from="s.whatsapp.net"][/start]

(2). rx [stream:features] rx [receipt_acks][/receipt_acks] rx [w:profile:picture type="all"][/w:profile:picture] rx [/stream:features]

(3) rx [challenge xmlns="urn:ietf:params:xml:ns:xmpp-sasl"]Ď]????+xoL?j??[/challenge]

(4) rx [success t="1374406602" xmlns="urn:ietf:params:xml:ns:xmpp-sasl" kind="free" status="active" creation="1373524843" expiration="1405060843"]?؋?y?w8+??,$??q?b[/success]

(4). a (Missing, dont know what message is : 00c00200)

(5). rx [presence from="s.whatsapp.net" status="dirty" xmlns="w"][/presence]

(6). rx [presence from="6221XXXXXXX@s.whatsapp.net" type="available"][/presence]

(7) rx  [iq from="s.whatsapp.net" id="4" type="result"] rx    [props version="1"] rx      [prop name="timeout" value="300"][/prop] rx      [prop name="library" value="0"][/prop] rx      [prop name="audio" value="0"][/prop] rx      [prop name="checkmarks" value="0"][/prop] rx      [prop name="newmedia" value="0"][/prop] rx      [prop name="image_max_kbytes" value="10240"][/prop] rx      [prop name="image_quality" value="75"][/prop] rx      [prop name="image_max_edge" value="800"][/prop] rx      [prop name="media" value="16"][/prop] rx      [prop name="broadcast" value="50"][/prop] rx      [prop name="max_subject" value="25"][/prop] rx      [prop name="max_participants" value="51"][/prop] rx      [prop name="max_groups" value="50"][/prop] rx    [/props] rx  [/iq]

(7) rx [iq from="g.us" id="3" type="result"][/iq]

(8) rx [iq from="6221XXXXXXX@s.whatsapp.net" id="5" type="error"] rx [error code="404" type="cancel"] rx [item-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"][/item-not-found] rx [/error] rx [/iq]

(9) rx [iq from="6221XXXXXX@s.whatsapp.net" id="1" type="error"] rx [error code="404" type="cancel"] rx [item-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"][/item-not-found] rx [/error] rx [/iq]

(10) rx [iq from="s.whatsapp.net" id="2" type="result"] rx [config xmlns="urn:xmpp:whatsapp:push" platform="gcm" id="APA91bHkf_tPGBb7AKRmp_ntdoPJB9rs2yG648Gvz7HeXpADOSz1Cj1sqhsihODwTvV5glIMcBkzAJO5aTjj0QlVUJSrAVwbuaeAVNS3rpNySe1hS2Bk6U7j8a5169JI1hZzN7CNvO7bKmoBA2ZN5N-XKZ8SGHRryw"][/config] rx [/iq]

(11) rx [iq from="s.whatsapp.net" id="6" type="result"][/iq]

(12) rx [iq from="s.whatsapp.net" id="7" type="result"][/iq]

Suspect : No 10 .. what kind of RX is that, and what TX is sent before it.

shirioko commented 10 years ago

I'm pretty sure that 8 & 9 are profile picture responses (the contacts didn't have a profile picture set) 10 could be triggered by sendGetClientConfig

assegaf commented 10 years ago

8, 9 yes.

7 a. is new, never found on whatsapi [iq from="g.us" id="3" type="result"][/iq]

who is is g.us ???? a robot ?

another point is, try to force stop the whatsapp , and re run. the challenge string did not sent again. so its true, its saved on client. tried to use old challenge string, nope, wrong/fail. and now lost the decyption key, cannt decrypt again

until sure about what tx is, nothing we can do.

still saving the tx after register, 6 of tx.

Edit: the id is using sequential, 1,2,3 ..etc not using time() like we used to use.

just googled about , platform="gcm" Google Cloud Messaging,

shirioko commented 10 years ago

g.us is the whatsapp groups server, probably a sendGetGroups response

assegaf commented 10 years ago

ok you are right, about g.us I tried with whatsapi again,

tx [iq id="3" type="get" to="g.us"] tx [list xmlns="w:g" type="participating"][/list] tx [/iq]

rx [iq from="g.us" id="3" type="result"][/iq]

about : sendGetClientConfig(), not sure what this means, said 404

tx [iq id="2" type="get" to="s.whatsapp.net"] tx [config xmlns="urn:xmpp:whatsapp:push"][/config] tx [/iq]

rx [iq from="s.whatsapp.net" id="2" type="error"] rx [error code="404" type="cancel"] rx [item-not-found xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"][/item-not-found] rx [/error] rx [config xmlns="urn:xmpp:whatsapp:push"][/config] rx [/iq]

Edit : my fault, its sendGetServerProperties(), it shows // also I didnot receive something like this below, even I has do sendGetClientConfig() :

@shirioko mind to tell me , what this tx means, it gotta to do with push, and sound ??

tx [iq id="3" type="set" to="s.whatsapp.net"] tx [config xmlns="urn:xmpp:whatsapp:push" sound="sound"][/config] tx [/iq]

I got RX Reply :

rx [iq from="s.whatsapp.net" id="3" type="error"] rx [error code="500" type="wait"] rx [internal-server-error xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"][/internal-server-error] rx [/error] rx [/iq]

shirioko commented 10 years ago

Wait is this traffic from the official client? How did you decrypt this?

assegaf commented 10 years ago

this one from official, just managed to decrypt rx. Edit : this one from official client https://github.com/venomous0x/WhatsAPI/issues/422#issuecomment-21308924

above you is from whatsapi, to try output something like sendGetClientConfig, sendGetServerProperties()

shirioko commented 10 years ago

Hmm not sure, WP7 uses an audio stream hack to push messages in the background, could have something to do with that

assegaf commented 10 years ago

its on whatsapi. I believe this is for wp7 only.

public function sendGetClientConfig() { $msgId = $this->createMsgId("sendconfig"); $child = new ProtocolNode("config", array("xmlns" => "urn:xmpp:whatsapp:push", "sound" => 'sound'), null, null); $node = new ProtocolNode("iq", array( "id" => $msgId, "type" => "set", "to" => static::WHATSAPP_SERVER ), array($child), null); $this->sendNode($node); $this->waitForServer($msgId); }

I still curious why I got this rx from official client. rx [iq from="s.whatsapp.net" id="2" type="result"] rx [config xmlns="urn:xmpp:whatsapp:push" platform="gcm" id="APA91bHkf_tPGBb7AKRmp_ntdoPJB9rs2yG648Gvz7HeXpADOSz1Cj1sqhsihODwTvV5glIMcBkzAJO5aTjj0QlVUJSrAVwbuaeAVNS3rpNySe1hS2Bk6U7j8a5169JI1hZzN7CNvO7bKmoBA2ZN5N-XKZ8SGHRryw"][/config]

shirioko commented 10 years ago

The node you received on the official client does seem to be a response to sendGetClientConfig according to it's response handler on WP7:

this.AddIqHandler(text, new FunXMPP.IqResultHandler(delegate(FunXMPP.ProtocolTreeNode node, string from)
    {
        FunXMPP.ProtocolTreeNode child2 = node.GetChild(0);
        FunXMPP.ProtocolTreeNode.Require(child2, "config");
        string attributeValue = child2.GetAttributeValue("id");
        this.EventHandler.OnClientConfigReceived(attributeValue);
    }, delegate(FunXMPP.ProtocolTreeNode node)

The one thing I'm not sure about is the "to" attribute in sendGetClientConfig:

    FunXMPP.ProtocolTreeNode node2 = new FunXMPP.ProtocolTreeNode("iq", new FunXMPP.KeyValue[]
    {
        new FunXMPP.KeyValue("id", text),
        new FunXMPP.KeyValue("type", "get"),
        new FunXMPP.KeyValue("to", this.Login.Domain)
    }, child);

this could be the reason we're getting 404 error

assegaf commented 10 years ago

ok no problem, once I know how to decrypt tx, I know exactly whats official client send.

shirioko commented 10 years ago

I was able to decrypt both key streams I captured from WhatsApi without any problem, but the official clients seem to cause a lot more trouble than I expected :/ Might be a good time to dust off MissVenom's source code and add back its TCP decryptor

assegaf commented 10 years ago

oh miss venom has sniffer 5222 before ? yeah, one point is about challenge string, I dont know why its only sent once after registration, after that it changed, and lost somewhere, relogin its not sent back again challenge string.

shirioko commented 10 years ago

Yep. Behold: https://github.com/shirioko/MissVenom/commit/55f503427673be5f6541a99a7eeb056a2eda3177

https://github.com/shirioko/MissVenom/commits/master 1.2.3.1 is the last version which had the TCP sniffer

The challenge string is only used for generating key streams, and I think that the official client reuses these streams. My workaround for this was to replace the pass through data with gibberish so the streams would get out of sync and the client would request a new challenge key on next connection.

assegaf commented 10 years ago

I hope so, reusing with the same keystream from the start, but at next login, it did not work, the chiper is completely wrong.

"pass through data with gibberish ", is that possible you completely in middle modifying packet sent back to android client. its mean miss venom not only sniffing also proxying.

shirioko commented 10 years ago

Yep, MissVenom uses proxies for all of its functionality (HTTPS, DNS and TCP) which allows you to modify any data that is passed through.

Are you resetting the key streams? Key streams change their state after each encrypted character, so you need to save it's state (e.g. serialize) after finishing decrypting the first connection and load it back (e.g. deserialize) when trying to decrypt a second connection.

assegaf commented 10 years ago

I will try old miss venom with tcp, kinda tired copy paste one by one packet on wireshark. and run on php each line of hex.

echo "#".y." bin[".$data."] \n"; $data = hextobin($data); echo "#".y." hex[".$data."] \n";

    $node = $reader->nextTree($data);

    $k = 0;

    while ($node != null) {

        $k++;
        echo "%".$k." ========= \n";
        echo ($node->nodeString("rx  ") . "\n");

        // check if celleng
        if (strcmp($node->tag, "challenge") == 0) {
            $challengeData = $node->data;
            echo "Found ChallengeData [".$challengeData."]  \n";
            echo "Found ChallengeData hex [".bintohex($challengeData)."]  \n";
            $reader->resetKey();
            $key = pbkdf2('sha1', base64_decode($password), $challengeData, 16, 20, true);
            echo "key [".$key."]  \n";
            echo "key hex [".bintohex($key)."]  \n";
            $inputKey = new KeyStream($key);
            $reader->setKey($inputKey);
        }

        $node = $reader->nextTree();
    }

not sure neither about saving keyStream.

assegaf commented 10 years ago

hi @shirioko

finally I able to to decrypt (actually not decrypt, first tx not encrypted), first tx is sent, and found very noticeable different between whatsapi and official client (android).

Whatsapi :

tx [stream:features] tx [receipt_acks][/receipt_acks] tx [w:profile:picture type="all"][/w:profile:picture] tx [/stream:features]

tx [auth xmlns="urn:ietf:params:xml:ns:xmpp-sasl" mechanism="WAUTH-1" user="XXXXXXXXXXX"][/auth]

Official Client :

tx [start to="s.whatsapp.net" resource="Android-2.10.222"][/start] tx [stream:features] tx [receipt_acks][/receipt_acks] tx [w:profile:picture type="all"][/w:profile:picture] tx [/stream:features]

tx [auth user="XXXXXXXXX" xmlns="urn:ietf:params:xml:ns:xmpp-sasl" mechanism="WAUTH-1"]?Z`?Z??KY?+??;?{???Oh???????vm???f=?G٢?9??[/auth]

see on tag auth official client, data, its sending something. on whatsapi its blank. thats why on official client server didnot give challenge string, maybe client send it,

any idea how to generate what to send at auth tag ? maybe its what you said serialize keystream ?

shirioko commented 10 years ago

http://xmpp.org/extensions/xep-0034.html WhatsApp uses it's own WAUTH-1 security mechanism, maybe it's the previously used challenge key?

assegaf commented 10 years ago

I hope its true, unfortunately I use another account to sniff this, so didnot know its password, cannt make keystream.

I will try with another account already saved password tonight to know what the heck is this [?Z`?Z??KY?+??;?{???Oh???????vm???f=?G٢?9??] exactly ..

Edit : one more, its using port 443 I dont know why whatsapp client sometime using 5222 sometime 443, I think its up to client decide which port being used. maybe some isp have firewall port,

assegaf commented 10 years ago

nope its not challenge key, try to make chiper with that, no lucks, I dont know what is that, looks like its needed as key/challenge to decrypt next message. we are stuck at here ...

Dynogic commented 10 years ago

@assegaf challenges are reused. Time to examine this code...

Dynogic commented 10 years ago

@assegaf @shirioko

AuthBlob has changed ... working on fixes. Challenges are now generated after auth success, which then becomes the challenge for the i+1 auth connection. So what @assegaf sees there is infact a challenge.

assegaf commented 10 years ago

that was fast, I examined the tcp for 2 days. ah I see,

any idea how client generate this challenge string?

Dynogic commented 10 years ago

@assegaf

I'm examining the source. And if I still can't figure it out, then I'll interpose the Java WA client to dump the ProtocolNodes.

Yes, I have already implemented it. I'm am now dropping my CodeRequest code in so I can get my password. xD