not working anymore

[makeev]

new param verifyFp=verify_kbg2qpav_kcD2KEdS_DR2W_4kAY_BnzD_rlugVyf6Sxs7 needed

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.65. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[makeev]

verifyFp is cookie s_v_web_id, but not working for me even with this param. Have no ideas for now

[sriramkumar1996]

I have changed the User-Agent, used verifyFp from s_v_web_id and also changed the goto url in browser.py to dynamically get _signature, but still not working. No ideas for now.

[harshall28]

Hi I am getting below trace: (just today only and was working till last night)

Converting response to JSON failed response is below (probably empty)

Traceback (most recent call last): File "/home/vidooly/ML/DeepLearning/Projects/user/harshall/practice/pytorch_prac/lib/python3.6/site-packages/TikTokApi/tiktok.py", line 50, in getData return r.json() File "/home/vidooly/ML/DeepLearning/Projects/user/harshall/practice/pytorch_prac/lib/python3.6/site-packages/requests/models.py", line 898, in json return complexjson.loads(self.text, **kwargs) File "/usr/lib/python3.6/json/init.py", line 354, in loads return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "scrap_no_watermark.py", line 26, in tiktoks = api.byHashtag(tag, count=count) File "/home/vidooly/ML/DeepLearning/Projects/user/harshall/practice/pytorch_prac/lib/python3.6/site-packages/TikTokApi/tiktok.py", line 236, in byHashtag id = self.getHashtagObject(hashtag)['challengeInfo']['challenge']['id'] File "/home/vidooly/ML/DeepLearning/Projects/user/harshall/practice/pytorch_prac/lib/python3.6/site-packages/TikTokApi/tiktok.py", line 272, in getHashtagObject return self.getData(api_url, b.signature, b.userAgent, proxy=proxy) File "/home/vidooly/ML/DeepLearning/Projects/user/harshall/practice/pytorch_prac/lib/python3.6/site-packages/TikTokApi/tiktok.py", line 54, in getData raise Exception('Invalid Response') Exception: Invalid Response

Is this a related issue? If yes when can we expect a resolution?

[makeev]

yes, its the same issue, no solution for now. I'll try to check deeply this evening

[tarkhil]

Same here, they seems to change signing algo again

[davidteather]

They seem to be detecting puppeteer / pyppeteer right now. There's a way to get around it with playwright, but there's no adaptation of playwright for python.

It seems that the puppeteer detection is only detecting the object routes such as getUserObject. For me I've changed the code and it supports bySound, because it never retrieves a sound object to get the music id, but not byHashtag or byUsername.

I'll work on this for a little bit longer and release what I have so far, but for the same level of functionality a python playwright rewrite might be a necessity. (unless I'm able to get around the object route detection)

Pyppeteer being detected on the hashtag page, it never loads signifying it's being detected.

[davidteather]

Alright I looked into it and I don't believe I'll be able to fix this without playwright (or another automation that isn't detected). I could interact with playwright through a server running on node, but that's not very practical.

[TimeToGoogle]

Did you try it with puppeteer-extra and maked use of their stealth package?

[davidteather]

@TimeToGoogle The current project uses pyppeteer-stealth which is based on puppeteer-extra-plugin-stealth. Still doesn't help.

[makeev]

Its not pyppeteer related, if url signed correctly - it's opening anywhere with same user-agent, even without cookies.

[davidteather]

Url doesn’t get signed correctly when tiktok detects it’s an automation script.

The automation software currently fails to load tiktok but does sign the urls but incorrectly. We can’t tell the browser to correctly sign them because it’s being detected.

[makeev]

But have you tried sign url in usual chrome console? I did and with no success.

[davidteather]

I haven’t tired in chrome console, but compare puppeteer on this version 1.1.4 vs the current version which uses playwright. Nothing major but changes for playwright was made.

The puppeteer version with extra stealth can’t sign urls correctly

The playwright version can sign urls that work.

[makeev]

yes, you a right, seems like i tested it incorrectly.

[Inzilkin]

Everything works fine through the Chrome console, but does not want to through pyppeteer.

[DeadlyMissile]

What did they change exactly? I can no longer access non watermarked videos

[davidteather]

They’re able to detect the automation software the library used, which they used to restrict displaying tiktok information from.

[DeadlyMissile]

@davidteather How does this affect grabbing non watermarked videos though? It is not automating anything

[davidteather]

Well to find the videos without a watermark it uses the automation software puppeteer to get a signature and find a redirect to the actual url.

[DeadlyMissile]

Shit we screwed then

[Inzilkin]

Who knows how to get video with him Id in api? Old api method sends white page

[Xodarap]

As a temporary fix, could you add a way for us to add the signature ourselves? (Ideally with some instructions on how to do so.)

This is the approach that this node-based API takes: https://github.com/szdc/tiktok-api

[dj2ball]

I have a successfully working android api based on an old apk - the endpoints from those old musical.ly apps are still active and lack the signature requirements of the updated app api. It doesn’t support signed login methods but those in this library work well and there is no need for a browser (pyppeteer etc) since the only check is for valid header info which you can generate from an android emulator. Would advise you take the api in that direction for now.

[Inzilkin]

I have a successfully working android api based on an old apk - the endpoints from those old musical.ly apps are still active and lack the signature requirements of the updated app api. It doesn’t support signed login methods but those in this library work well and there is no need for a browser (pyppeteer etc) since the only check is for valid header info which you can generate from an android emulator. Would advise you take the api in that direction for now.

can you help me with getting signatures from the app? tg - @inzilkin

[dj2ball]

If you use an old app version they lack the code to authenticate using X-Gorgon and Khronos (the new mobile signatures) and the signature being discussed here is for the desktop version only. The old apps use CP, AS and MAS but it does not appear to be reliably enforced on the client ( try reversing the Java library code using Jadx and you’ll see the code allows placeholder values to be acceptable)

[dibashthapa]

Should I have to decompile the app to get the CP , AS and Mas ?? I tried to find those params in api but I couldn't find any.

[derekcamarillo]

I have the same problem with v3.2.3 on Python3.8 Is this fixed?

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/TikTokApi/tiktok.py", line 52, in getData return r.json() File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 898, in json return complexjson.loads(self.text, **kwargs) File "/usr/local/lib/python3.8/json/init.py", line 357, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "tiktok_api_scraper.py", line 8, in user_data = api.getUserObject(user_id) File "/usr/local/lib/python3.8/site-packages/TikTokApi/tiktok.py", line 327, in getUserObject return self.getUser(username, language, proxy=proxy) File "/usr/local/lib/python3.8/site-packages/TikTokApi/tiktok.py", line 336, in getUser data_chunk = self.getData(api_url, b, proxy=proxy)['userInfo'] File "/usr/local/lib/python3.8/site-packages/TikTokApi/tiktok.py", line 56, in getData raise Exception('Invalid Response') Exception: Invalid Response

[davidteather]

I've been working on this I've managed to get the user object route working again as it looks like TikTok is changing the way they do this.

I've found a relevant issue here on TikTok detecting puppeteer. I've been trying to work around this with little success.

[olitomas]

Related to this? 🤔 https://github.com/digitalhurricane-io/puppeteer-detection-100-percent

[davidteather]

https://github.com/davidteather/TikTok-Api/releases/tag/V3.3.0