Status 200 is always thrown

[tmichel124]

Currently when I run the trending function, it never returns a status 0. I believe that the signature found via selenium is not a totally valid one, and if you go to the tiktok trending website with a headed browser you can see the videos never really load

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.96. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[davidteather]

Thanks for pointing out the issue. Yeah the headed browsers don't work on my computer either.

I'll start to work on a fix, but honestly not sure I can as easily, because a lot of the API is based on headed browsers.

[Lem0nTree]

Yup same here, until yesterday it worked as hell, now the api for hashtag doesn't work. Looks like it can't retrieve the video list.

[davidteather]

I've been able to get around the not loading any video on the tiktok trending page, but it seems like the requests library isn't working anymore. I'd rather not use selenium for everything, but it's looking like I might have to.

[Lem0nTree]

P.s happy halloweed man :D

[davidteather]

Thanks, for some reason it's snowing on Halloween for me.

[tmichel124]

what did you do to get around the trending page loading?

[davidteather]

chromeProfile = webdriver.ChromeOptions()
chromeProfile.add_argument("--disable-automation")
chromeProfile.add_experimental_option("excludeSwitches", ["enable-automation"])
chromeProfile.add_argument("--proxy-server={0}".format(proxy.proxy))
chromeProfile.add_experimental_option("mobileEmulation", {"deviceName": "Galaxy S5"})

if headless == True:
    chromeProfile.add_argument('headless')
driver = webdriver.Chrome(chrome_options=chromeProfile)

Instead of the firefox driver on lines 31 in tiktok.py I had to switch to chrome driver. I'll release a version that works, when I find a solution for the other functions not working.

[tmichel124]

thanks so much! that will help in the meantime

[tmichel124]

I was trying this code and it still looks like all the responses are status 200 when I look at the proxy har, can you confirm this is still working for you?

It seems to work correctly when I comment out the headless argument line, though!

[davidteather]

I didn't test it as a headless browser honestly, I'm just trying to get the API to work first. It might not work on headless right now.

[Lem0nTree]

I checked other tiktok api on github, neither one of them seems to work :(

[davidteather]

Yeah I’ve also been struggling to get any type of response. The most I’ve gotten is the top 2 from trending but that’s it.

[Lem0nTree]

So no hope for make them work in future? I was working on a project with those api since september :(

[Lem0nTree]

Btw you have done a really great job ,if you have a paypal or so i wih to contribute to your work!

[davidteather]

I hope to make it work in the future, however no guarantees. I have a solution in mind, but it might take a while and not be as fast as before.

Just added a sponsorship button to the repository. Here's the link anyways. Also thanks for even considering to donate. I never really thought this api would get many downloads or even someone who wanted to donate.

[Lem0nTree]

You made a great work, i learned a lot with your code! I will make the donation this week, let me know if you also have a LTC address

[davidteather]

Thanks man!

LTC: MCFSPnGDiS5x7GMYBgDrfzBHVuR6rDwFek

[davidteather]

I released a new PyPi version that supports the downloading video by url method.

Here is the release.

Still working on fixing the other methods in my free time.

[Lem0nTree]

Here a small gift for your amazing work 👍

Keep up the great work!

[davidteather]

@theparcel Thanks so much!

[Parallaps]

Are there any updates?

[davidteather]

Unfortunately not really. I’m attempting to use selenium to get the divisions of the tiktok videos but it seems like that’ll even be limited because of their protection against this stuff.

[mjaensch]

I also tried a bit to play around with selenium to get your API working again. It seems like the script https://www.tiktok.com/akam/11/1f5e5639 is responsible for detecting selenium. When you run it through a deobfuscator you can see a lot of different approaches to detect selenium: https://pastebin.com/r21Jjkxm Interestingly the script is only loaded when I open a tiktok page in selenium.

[Lem0nTree]

This fix work for other sites that detect selenium: https://intellipaat.com/community/5490/can-a-website-detect-when-you-are-using-selenium-with-chromedriver

https://stackoverflow.com/questions/33225947/can-a-website-detect-when-you-are-using-selenium-with-chromedriver

[Lem0nTree]

Updated to the latest version, and added the chromedriver stuff; but got this error: DevTools listening on ws://127.0.0.1:55758/devtools/browser/54bd65b9-db59-4209-855a-5e134c8fbba0 [1118/145356.352:INFO:CONSOLE(1)] "version:0.1.7", source: https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/_app.js (1) [1118/145356.611:INFO:CONSOLE(1)] "Mixed Content: The page at 'https://www.tiktok.com/trending?lang=en' was loaded over HTTPS, but requested an insecure image 'http://p16-tiktokcdn-com.akamaized.net/obj/tiktok-obj/1617014531310594.JPEG'. This content should also be served over HTTPS.", source: https://verify-sg.byteoversea.com/static/pc_slide.js (1) Found signature, continuing. Numbers of video to download: 10 [1118/145359.570:INFO:CONSOLE(0)] "The resource https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/home.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriateasvalue and it is preloaded intentionally.", source: (0) [1118/145359.574:INFO:CONSOLE(0)] "The resource https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/explore.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriateasvalue and it is preloaded intentionally.", source: (0) [1118/145359.574:INFO:CONSOLE(0)] "The resource https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/about.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriateasvalue and it is preloaded intentionally.", source: (0) Hashtag to target: meme [1118/145440.214:INFO:CONSOLE(1)] "version:0.1.7", source: https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/_app.js (1) [1118/145440.532:INFO:CONSOLE(0)] "Mixed Content: The page at 'https://www.tiktok.com/tag/memes?lang=en' was loaded over HTTPS, but requested an insecure image 'http://p16-tiktokcdn-com.akamaized.net/obj/tiktok-obj/1617014531310594.JPEG'. This content should also be served over HTTPS.", source: https://www.tiktok.com/tag/memes?lang=en (0)

[davidteather]

What code are you running?

[Lem0nTree]

Added you to my project on github, to show you the code :)

[Lem0nTree]

Still didn't find a solution, i looked around on the web for possible workaround but can't make it work again :(

[davidteather]

What chromedriver version are you using? I don't get that error on V78

[dombrocarl]

have you been able to make this api works?

I can pay you 1000 dollars if you make that works :)

[davidteather]

@dombrocarl

The api works right now. It's not the same as before TikTok changed their website. It does not work on a headless browser. I've been attempting to make it the same as before they changed their website, but I don't think I'll be able to make it exactly like that.

Feel free to ask any questions you have using the new slightly different API.

[davidteather]

I think I might have found a work around that works for some reason. I currently have the trending method working for this api, but not the other ones yet.

It gives the json response like it used to before TikTok changed their website.

[davidteather]

@theparcel @tmichel124 @Parallaps @mjaensch @dombrocarl

I have a fix for this API that will be released to PyPi in about half an hour. I'm just drafting the release notes and it should be up.

[reteps]

The issue popped back up.

My changes:

chromeProfile.add_argument('--ignore-certificate-errors')
chromeProfile.add_argument('--ignore-ssl-errors')
# later in trending
t = 0.5
if data["statusCode"] == 0:
    ...
elif data['statusCode'] == 200:
    t *= 1.5
    print(f'Denied. sleeping for {t} sec')
    time.sleep(t)
    print('now trying again')

Gives me

...
now trying again
Denied. sleeping for 145.96 sec

It was working earlier, but suddenly stopped. @davidteather

[davidteather]

@reteps you might want to try having no delay. The only reason this works is through spamming the servers until it responds.

[reteps]

It works with no delay, but that is pretty sus

[davidteather]

@reteps yeah it’s an unofficial api and for some reason you need to spam their servers to get a request