search

tomer8007 / kik-bot-api-unofficial

Python API for writing unoffical Kik bots that act like humans
MIT License
128 stars 77 forks source link

SafetyNet #58

Closed Jaapp- closed 5 years ago

Jaapp- commented 6 years ago

I've been having the safetynet issue too. Here's steps to reproduce for me:

  1. Use new device_id and android_id to isolate the issue
  2. Run register bot to create a new account
  3. Account creation (or login) succeeds
  4. The follow messages is received:
    <kik timestamp="1530632332578" app="all" push="false" qos="false" hop="true" />
    <request r="false" xmlns="kik:message:receipt" d="false" />
    <xiphias-mobileremote-call method="TriggerSafetyNetCheck" service="mobileremote.antispam.safetynet.v1.MobileSafetyNet">
        <body />
    </xiphias-mobileremote-call>
</message>
  1. A couple of seconds later the connection is closed from Kik's end.

I haven't recorded my phone for account creation yet.

gituserdxd commented 6 years ago

@Jaapp- the way I fixed it was using the same device ID as my emulator which already has kik apk installed on it. I'm not 100% sure how this safety net thing works but I think it checks to see if that certain device ID has logged in with an APK before. If it has it probably whitelists it if not then it blacklists. Just my guess from what I experimented. You should try it with device ID of your emulator or phone's if you have it rooted so you can easily change it in case they decide to blacklist it.

gituserdxd commented 6 years ago

Also I recorded my phone for login but it never triggered safety net on it. Only on PC it did

tomer8007 commented 6 years ago

Also in case this fails I think we can try just going back to kik's older version like it was before. We know it doesn't send the SafetyNet message there, but it does require captchas on first login.

gituserdxd commented 6 years ago

I think the hardcoded timestamp used to generate "CV" value is what's causing this. Think about it we had the timestamp for July 1 2017 for 11.1 then we changed it to 13.4. Kik would obviously get suspicious and wonder how we logged in/established session on two different versions at the same time so they blacklist your device ID

dexterdj commented 6 years ago

You have to request a "Nonce" ID from kik server, and use this nonceID to emulate the "phone compatibility check process" from SafetyNet (It's a Google Play service).

Doc here https://developer.android.com/training/safetynet/attestation

Sample application to send SafetyNet request from Android: https://github.com/googlesamples/android-play-safetynet/tree/master/client/java/SafetyNetSample

Sequence architecture to implement:

PythonBot ------------requestGetNonce ----------> Kik server PythonBot <----------NonceResponse -------------

PythonBot -------TCP/IP-sending "nonce" value ----> Android app in Real device

Android app in Real device: Using of official SafetyNet API to autenticate the phone in GooglePlay, and receive a "JwsResponse".

(At this moment, Google will also send the JwsResponse to Kik Servers, it's safetynet process ...)

Android app -------- JwsResponse--------------> PythonBot PythonBot -------- JwsResponse--------------> KikServer

(Kik server receive the same JwsResponse from our PythonBot, and from Google. All is OK)

For now, i developped the android java gateway to send the requests to google, but i only tested on NOX emulator. The JwsResponse contains JSON data with phone informations, emulator is detected.

The process have to be reproduced from real devices.

I will test this way latter ...

(If anyone interested to work on this process with me, i can share my work).

Skype: dexter.powal

Cdt,

tomer8007 commented 6 years ago

From my experience with SafetyNet in the past, this is something that's hard to emulate without a real device. I think we should get back to version 11.

Jaapp- commented 6 years ago

Sounds like a good idea. Though in the future it would indeed be nice if we could find a way to simulate this for a device python. I've seen an xposed module to pass safety net, maybe we can rewrite that.

tomer8007 commented 6 years ago

Is it up-to-date? I heard of some methods to bypass SafetyNet but I think Google keeps blocking them as soon as they understand how they work. Can you post the link?

gituserdxd commented 6 years ago

@tomer8007 I'm able to login fine even with the safety net message showing. Also I figured out how to login with current timestamp not the hardcoded one that translates to July 1 2017. If you want I could give you the java code for it and you can transcribe it to python or I could do it for you and upload a pull request

tomer8007 commented 6 years ago

This is really great. How were you able to use the current timestamp?

gituserdxd commented 6 years ago

@tomer8007 it took a lot of digging through kik's code to see how they did it and when I figured it out I was easily able to implement it in my java bot. Now goal is to make the same in python

tomer8007 commented 6 years ago

So basically what's needed to be changed in the current python implementation?

gituserdxd commented 6 years ago

@tomer8007 Kik has it's own custom class for generating timestamps so that needs to be added and that timestamp needs to be hashed with our JID and kik's hmac to generate a checksum for cv value. Idk if python supports multi-threading or volatile methods/fields because that's what that class uses I'm better at java than python so I'm not sure if there is an equivalent of that in python. If you have telegram lmk and I'll send the java code there

tomer8007 commented 6 years ago

@gituserdxd Yes, the timestamp is already hashed and used to generate the cv value in the current code. Is that what you meant? Also, what do you mean by a custom class for generating timestamps? Can't it just be the current UNIX timestamp?

Jaapp- commented 6 years ago

Ah yeah, SafetyNet is going to be a pain. I reverted back to 11 for my client, I suggest we do this for the API too.

tomer8007 commented 6 years ago

@Jaapp- Yes, that's already done.

Jaapp- commented 6 years ago

@tomer8007 Ahh, seems I'm lagging behind.

tomer8007 commented 6 years ago

Update I think I solved the disconnection issue by removing the line self.loop.call_later(20, self.loop.stop) which basically schedeled a disconnection for 20 seconds in the future. If this indeed works, it means the disconnection was in fact from our end and didn't have anything to do with SafetyNet.

@gituserdxd Also could you please post the code that generates the timestamp? It could be really helpful. You can also send it to me and I will transform it to python.

hunterbeach commented 6 years ago

I removed that line but I am still getting disconnects.

tomer8007 commented 6 years ago

Hmm. are these unexpected disconnects without any log? What does wireshark show? who sends the FIN/RST?

hunterbeach commented 6 years ago

Yeah, they are unexpected. It runs for a random amount of time, then isn't able to send messages anymore, I assume because it disconnected. Then I restart it and it connects again. I can try to get a wireshark capture but because it is random, it might be hard to capture.

tomer8007 commented 6 years ago

Does it happen after a short period of time?

hunterbeach commented 6 years ago

Yeah, it's pretty random but usually between a day or two, then I get 

AttributeError: 'KikConnection' object has no attribute 'send_chat_message'
tomer8007 commented 6 years ago

Oh. A day or two is actually a long period of time. And you even have an exception printed? That's good. I think I know the cause of it. Just look for a line that tries to call it.

Are you on the latest commit?

hunterbeach commented 6 years ago

The line that tries to call it is line 236 in client.py

This is the entire stacktrace:

[2018-07-19 06:15:13,038] ERROR  (thread Kik Connection): Exception in callback
KikClient._on_new_data_received(b' ')
handle: <Handle KikClient._on_new_data_received(b' ')>
Traceback (most recent call last):
  File "C:\Python36\lib\asyncio\events.py", line 145, in _run
    self._callback(*self._args)
  File "C:\Python36\lib\site-packages\kik_unofficial\client.py", line 236, in _o
n_new_data_received
    self.loop.call_soon_threadsafe(self.connection.send_chat_message, b' ')
AttributeError: 'KikConnection' object has no attribute 'send_chat_message'

The interesting thing is the comment above that line which is: # Happens every half hour. Disconnect after 10th time. Some kind of keep-alive? Let's send it back

I might not be on the latest commit, I'm not sure so I'll check later and update if not

Also, like I said it is really random. I just restarted the program at my last message which was 2 hours ago and it already disconnected again.

tomer8007 commented 6 years ago

Listen, I'm pretty sure that I fixed this bug already on the latest commit. Just use it.

hunterbeach commented 6 years ago

Alright, I just updated to the latest commit and restarted, I'll comment back if the issue persists

hunterbeach commented 6 years ago

Well, on the latest commit and this time it disconnected I assume, but with no stacktrace at all. Still was able to reconnect after restarting the program.

tomer8007 commented 6 years ago

Umm, can you see in wireshark which side finishes the connection?

hunterbeach commented 6 years ago

Not really, but here is a pcap file that should contain it disconnecting from the server. I left it running until it disconnected then filtered (very painfully since the file was huge and had to be loaded into memory to filter) everything that didn't go to port 5223. Packet 22 looks like Kik sends the fin first.

KikDisconnect.zip

gituserdxd commented 6 years ago

Hey @tomer8007 I uploaded the code for kik's current timestamp generation for session

tomer8007 commented 6 years ago

@gituserdxd That's really great but where did you upload it to? @theman00011 Yes, it appears that Kik sends the FIN first in packet 22. Does it happen after a constant amount of time? Can you post the debug logs too?

hunterbeach commented 6 years ago

@tomer8007 I don't believe it happens after a constant time but I'll see if I can find out. Also, which debug logs are you referring to?

This is where he uploaded it to https://github.com/gituserdxd/KikCustomTimeStuff

gituserdxd commented 6 years ago

Check my recent repo @tomer8007

tomer8007 commented 6 years ago

@theman00011 I mean the logs that the client produces when the debug level is set to DEBUG instead of INFO. @gituserdxd Oh cool, I'll see if I can translate that to python, thanks

hunterbeach commented 6 years ago

@tomer8007 I figured that out, I already started logging, I'm just waiting for it to disconnect again.

hunterbeach commented 6 years ago

@tomer8007 Well the debug log didn't yield anything interesting. The last lines before it disconnected were just read receipts working normally and then

[2018-07-25 08:41:34,777] DEBUG  (thread Kik Connection): [!] Main loop ended.

Here's the full log with messages and stuff cut out as well as usernames scrubbed. (It might be a good idea to not log the node and keys in a future update so that someone doesn't accidentally compromise their account.) You can see it was receiving read receipts right up until the main loop ended.

KikLog.txt

hunterbeach commented 6 years ago

Any update on this?

tomer8007 commented 6 years ago

@theman00011 Hopefully I could reproduce this problem in the next days or so. Sorry for that taking time

tomer8007 commented 6 years ago

My current guess: an asyncio problem

tomer8007 commented 6 years ago

@theman00011 How about SafetyNet? Do you have problems with it too? Do you see capthcas on login?

Regarding the unexpected disconnect, actually, since kik are the side that's ending the connection I'm not sure whare we are doing wrong. Maybe that's due to inactivity. What do you think?

I also see not-implemented xmlns exceptions in your log. Maybe that could be the cause. I commited a change that replaces this NotImplemented exception with logging.

rmcc3 commented 6 years ago

@tomer8007 Using node, I seem to get "Not Authorized" upon login if the program gets ran outside the PyCharm debugger, for some odd reason. The issue reproduces in a new environment also. No problems with login running it inside the debugger, strangely.

I'm running it the same way; the only difference is that the debugger is attached when none of the issues occur. Either I'm missing something, or it might be something to look into more.

hunterbeach commented 6 years ago

@tomer8007 Yeah, I get captchas almost every time I login with the node, running from the command line.

I'll try to get time to capture a disconnect with the new commit. It's possible it is some sort of timeout but then we should be able to correlate it disconnecting after X amount of time.

tomer8007 commented 6 years ago

@rmcc3 That's just too strange. I'm running it without a debugger fine (only with a CAPTHCA). Did you try logging all the communication so you could tell the difference?

@theman00011 I get captchas only when logging it without a node. Logging in with a node seems to work smoothly.

hunterbeach commented 6 years ago

@tomer8007 For me sometimes after a random disconnect, I will restart the program and try to login with the node, and I get a non-authorized error so I have to remove the node and login with a user&pass and complete the captcha and login that way. Then I can login a few more times with the node before I have to do the captcha again.

rmcc3 commented 6 years ago

@tomer8007 it works fine with captcha. I'm getting the error on both debugger and CLI now. Seems to only work randomly.

leotorrez commented 6 years ago

I have this same issue. Node works for one or two days ansago then I need to log in without a node and complete a captcha to reactivate said node

tomer8007 commented 6 years ago

@leotorrez @theman00011 This CAPTCHA was caused most likely because we didn't use the most up-to-date version of the kik APK.

I have commited a newer version in device_configuration.py and now it doesn't require captcha on login anymore.

leotorrez commented 6 years ago

@tomer8007 THANKS A LOT!!!! I will jump right away to check on that.