davidteather / TikTok-Api

The Unofficial TikTok API Wrapper In Python
https://davidteather.github.io/TikTok-Api
MIT License
4.69k stars 952 forks source link

Accessing The Android/IOS TikTok Calls #120

Open tarkhil opened 4 years ago

tarkhil commented 4 years ago

Sorry for reopening closed issue, but that API call does exists.

I've tested https://rapidapi.com/logicbuilder/api/tiktok and it returns list of followers (but not of following).

Unfortunately, their uniform and well-developed API has nothing in common with cryptic TikTok API, so I still cannot guess how to do it without 3rd party sites.

issue-label-bot[bot] commented 4 years ago

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

dj2ball commented 4 years ago

I’m pretty sure that is a mobile end point, so you would need to reverse engineer the calls and authentication to that. You can test it by opening tiktok on your phone, check your not logged in and click on someone’s “followers” and it returns the list

tarkhil commented 4 years ago

I'll try to install Android emulator and look, but I'm not sure when I'll have the time. Maybe someone with higher reverse engineering skill is reading this issue?

dj2ball commented 4 years ago

I have rebuilt mobile endpoints from old versions of the Android APK using Charles Proxy, it's mostly straightforward. The question is what version of the app introduced this feature and what signature encryption is used. I'll take a look later

tarkhil commented 4 years ago

@dj2ball AFAIK signature encryption is handled just fine by existing TikTokApi code. What's straightforward for you means installation of unknown soft for me, I'm not a mobile developer. Thank you in advance for taking a look.

davidteather commented 4 years ago

I'm open to adding this feature if anyone is able to reverse engineer it. Hopefully they use the same signature generating methods, but I feel that's unlikely.

dj2ball commented 4 years ago

I’ve had good success using Nox Android Emulator. Charles Proxy is paid but has a free trial and you can use it to intercept traffic from your mobile app as a man in the middle proxy - good example here:

https://deliveroo.engineering/2018/12/04/how-to-use-charles-proxy-to-rewrite-https-traffic-for-web-applications.html

If you are more comfortable with python mitm-proxy is I think the free package that does similar things.

You will want to use Android OS Version 7 or earlier in your emulator as after that version there are restrictions on installing 3rd party SSL certificates which you need to do to read HTTPS traffic from TikTok.

On Jun 6, 2020 at 2:43 pm, <tarkhil (mailto:notifications@github.com)> wrote:

@dj2ball (https://github.com/dj2ball) AFAIK signature encryption is handled just fine by existing TikTokApi code. What's straightforward for you means installation of unknown soft for me, I'm not a mobile developer. Thank you in advance for taking a look.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/davidteather/TikTok-Api/issues/120#issuecomment-640063327), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AHNHPE5AWTBKFCLB6RHAQHDRVJBXPANCNFSM4NU6NC7Q).

dj2ball commented 4 years ago

I'm pretty sure you would need different authentication. When I looked at this before, the current implementation of the web api uses verifyFp and _signature as authentication in its https request, the mobile app uses Khronos and X-Gorgon.

I've done a bit of decompiling of the Java APKs and from what I can see, Khronos is essentially a Unix timestamp with query parameters passed to it but Gorgon is heavily encrypted. There is some info online about a few folks who have built generators for this (e.g. https://github.com/SebastienWae/tiktok-gorgon-bridge) although I've yet to get it working successfully (I can get it installed on android and communicate with it via cURL but currently the signatures aren't accepted. It's possible I need to try a different app version as it may get updated.

INFINITEY6622 commented 4 years ago

@dj2ball Hey please let me know after you tried a diffrent app version if it worked or not thanks. @tarkhil I suggest you using Memu andriod version 7 with Xpossed framework download the Sslunpinning moudle from Xpossed framework and install it with that you can ssl unpin the tiktok app and for capturing data/sniffing i suggest you using fiddler its a free sniffing tool.

dj2ball commented 4 years ago

I checked a couple of my older Android APKs and there is no follower endpoint without logging in. On the current iOS version the endpoint is there.

We would need current signature generation for the Khronos and X-Gorgon headers to be able to access the endpoint, at a minimum.

davidteather commented 4 years ago

I just looked into generating those parameters and this repo seems promising. I can’t look into it too much right now but I’ll be able to look into it this weekend. Generating Parameters

dj2ball commented 4 years ago

Sounds good, that repo does look promising - if you decompile the Android tiktok api the code for generating X-Gorgon and Khronos via Leviathan it looks pretty close to what is in that repo

dj2ball commented 4 years ago

I believe Leviathan has to be called in the Java Native Environment (JNI) and is part of the libcms.so binary.

The code posted by David seems to be a good approximation of the steps to generate the inputs for the leviathan call, what I'm wondering is then whether you need something like the repo below to hook leviathan in the JNI and generate the return?

See below:

https://translate.google.com/translate?hl=en&sl=auto&tl=en&u=https%3A%2F%2Fgithub.com%2FAeonLucid%2FAndroidNativeEmu%2Fblob%2Fmaster%2FREADME_cn.md

Unfortunately Java is a little away from my wheelhouse so at this point I'm following logic and intuition rather than a solid understanding of Android development.

This thread summarises some good discussion on it:

https://github.com/szdc/tiktok-api/issues/120 including this summary:

"For anyone struggling with generating XGorgon/XKhronos combination:

XGorgon is generated by concatenating four strings:

Your string should now be 32 * 4 digits long. Set X-Khronos to the current time and pass this value + the string you just generated to the Leviathan encryption algorithm.

This algorithm is extremely complex and can't be easily reverse-engineered. It has lots of security checks to see that your phone isn't rooted, using any emulator, using a proxy/vpn, etc... . It also makes a lot of syscalls which makes it hard to emulate. However, it is possible to reverse with enough determination (Took me around 2 weeks).

The value that is returned by leviathan then just needs to be Hexed, and this is your X-Gorgon value."

davidteather commented 4 years ago

There’s a list of what seems to be most of the apk calls that the tiktok app calls on the website listed below. Just scroll to the tiktok research segment then there’s a pdf called apk_calls or something like that.

https://penetrum.com/research

rabea98 commented 4 years ago

Hi, so i have been digging through the tiktok traffic on ios through fiddler, and i saw once you press the following list, this request gets made:

https://api2-16-h2.musical.ly/aweme/v1/user/following/list/?version_code=8.8.0&language=en&pass-region=1&app_name=musical_ly&vid=60935E2D-0811-4D7F-912F-4935AF6BCC6D&app_version=8.8.0&carrier_region=IL&is_my_cn=0&channel=App%20Store&mcc_mnc=42503&device_id=6808215953622566406&tz_offset=10800&account_region=IL&sys_region=IL&aid=1233&screen_width=1242&openudid=c3d30ddcb7b810ec798c94cd55a5c88ff457f3d8&os_api=18&ac=WIFI&os_version=13.3.1&app_language=en&tz_name=Asia/Jerusalem&device_platform=iphone&build_number=88005&device_type=iPhone8,2&iid=6852069449127102213&idfa=00000000-0000-0000-0000-000000000000&offset=0&user_id=6830471606386279430&source_type=2&count=20&max_time=1596061205&mas=017df9435b048f86ac28a274543ac46df5e20e0ecff32123a58287&as=a1b53f12c4017f76116532&ts=1596061204

### is it possible to save the .json file that is returned from this URL ? the problem is that it can be requested only once by the tiktok app, if you request it again , the returned .json file is empty

here's a screenshot of the .json from fiddler

image

I'm interested in the following tab from .json file, as it has a list of 20 users

alyfreym commented 4 years ago

Hi guys!. its possible get tiktok music endpoints?

davidteather commented 4 years ago

@alyfreym what type of music end points?

alyfreym commented 4 years ago

@alyfreym what type of music end points?

I would like to receive music requests that are used in a mobile application. I know that all requests are encrypted x-gorgon, x-khronos. Do you have any information about these algorithms how to do it?

davidteather commented 3 years ago

@alyfreym kinda a late response, but this api already has some music endpoints. Still haven't figured out any algorithms to do this that are open source. If anyone has links to any comprehensible working projects that would be nice.

davidteather commented 3 years ago

Also might want to check out https://github.com/augustgl/tiktok_source

elblogbruno commented 3 years ago

https://www.youtube.com/watch?v=RxkLFAGetVQ I saw this video today, where he edits his tiktok profile with the videos current views and likes!

davidteather commented 3 years ago

@AdKT36 If you want to post stuff don't post sketchy site links

davidteather commented 3 years ago

This might be interesting seems to get mobile endpoint data you do need a device_iid and iid but for my internet I just typed in random stuff and it worked. Not sure if that would work at scale though.

megapro17 commented 3 years ago

However, it is possible to reverse with enough determination (Took me around 2 weeks).

Why you not posted any result?

AlessandroSpallina commented 3 years ago

Any chance to support the retrieving of the followers list @davidteather? From the website this information is now available! Cattura

davidteather commented 3 years ago

Any chance to support the retrieving of the followers list @davidteather? From the website this information is now available!

It’s only available while logged in which is not the focus of the package right now. I’ll look into implementing more user functions in the future when I have more time available

pyalchy1 commented 2 years ago

Lol, Noobs

pyalchy1 commented 2 years ago

Just set up a Fake AP and forward the traffic through a reverse proxy to your router. That's the only reliable way to intercept android application calls.

Lackaday9 commented 2 years ago

I've done a bit of digging and found some repos that contain old musical.ly endpoints.

https://github.com/tolgatasci/musically-tiktok-api-python/blob/master/api.py https://github.com/szdc/tiktok-api/blob/27d9af2ccba7404d1703b2de666618e8837438b9/src/index.ts

I've tried some of the endpoints. Some like "video metadata" and "comments" work fine with no headers and do contain a lot more fields than the web version, however more interesting ones like the following/followers lists or even the user's post list just seem to respond with empty 200 responses. Even if I use the same headers they use. I suspect something has changed and now they use other headers/params. I feel this because the URLs themselves are returning a 200. If I try use a fake URL like https://api2.musical.ly/aweme/v1/bogus/post/?user_id=etc then I get this response: {"status_code": 1, "message": "Url does not match"}

I'm not sure where to go from here. Any ideas on finding the right headers/params? I feel x-gorgon and x-khronos would be important. This code claims to generate them, but is incomplete (https://github.com/bloodyev/x-gorgon-x-argus-x-ladon/blob/main/xgorgon_orig.py), what is data? I don't use Telegram or Skype so I can't contact him either.

sanernecro commented 8 months ago

Hi, so i have been digging through the tiktok traffic on ios through fiddler, and i saw once you press the following list, this request gets made:

https://api2-16-h2.musical.ly/aweme/v1/user/following/list/?version_code=8.8.0&language=en&pass-region=1&app_name=musical_ly&vid=60935E2D-0811-4D7F-912F-4935AF6BCC6D&app_version=8.8.0&carrier_region=IL&is_my_cn=0&channel=App%20Store&mcc_mnc=42503&device_id=6808215953622566406&tz_offset=10800&account_region=IL&sys_region=IL&aid=1233&screen_width=1242&openudid=c3d30ddcb7b810ec798c94cd55a5c88ff457f3d8&os_api=18&ac=WIFI&os_version=13.3.1&app_language=en&tz_name=Asia/Jerusalem&device_platform=iphone&build_number=88005&device_type=iPhone8,2&iid=6852069449127102213&idfa=00000000-0000-0000-0000-000000000000&offset=0&user_id=6830471606386279430&source_type=2&count=20&max_time=1596061205&mas=017df9435b048f86ac28a274543ac46df5e20e0ecff32123a58287&as=a1b53f12c4017f76116532&ts=1596061204

### is it possible to save the .json file that is returned from this URL ? the problem is that it can be requested only once by the tiktok app, if you request it again , the returned .json file is empty

here's a screenshot of the .json from fiddler

image

I'm interested in the following tab from .json file, as it has a list of 20 users

hello, can you still sniff for ios tiktok?

Fabccc commented 8 months ago

Done some research for tiktok on web for uploading. can't get further than https://www.tiktok.com/api/v1/web/project/post/. i'm getting an

{"log_pb":{"impr_id":"20240107162533067FBF6BE998946340B6"},"status_code":4,"status_msg":"Server is currently unavailable. Please try again later."}

I tried the upload process on web using Edge, the URL requires an msToken (I don't know how to generate it). I'm using your project as a local callable API for my Java program. In the HAR file, i'm seeing multiple encrypted header (encoded as base64) image

Despite a few days of efforts, I can't get it to work on this part. Also, it seems that the UploadID returned by their API when creating a project is not the same as the uploadId they send to an AWS storage: image image

idontneedgithub commented 2 weeks ago

someone has any updates on the followers list?