search

davidteather / TikTok-Api

The Unofficial TikTok API Wrapper In Python
https://davidteather.github.io/TikTok-Api
MIT License
4.92k stars 986 forks source link

Status 200 is always thrown #18

Closed tmichel124 closed 4 years ago

tmichel124 commented 5 years ago

Currently when I run the trending function, it never returns a status 0. I believe that the signature found via selenium is not a totally valid one, and if you go to the tiktok trending website with a headed browser you can see the videos never really load

issue-label-bot[bot] commented 5 years ago

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.96. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

davidteather commented 5 years ago

Thanks for pointing out the issue. Yeah the headed browsers don't work on my computer either.

I'll start to work on a fix, but honestly not sure I can as easily, because a lot of the API is based on headed browsers.

Lem0nTree commented 5 years ago

Yup same here, until yesterday it worked as hell, now the api for hashtag doesn't work. Looks like it can't retrieve the video list.

davidteather commented 5 years ago

I've been able to get around the not loading any video on the tiktok trending page, but it seems like the requests library isn't working anymore. I'd rather not use selenium for everything, but it's looking like I might have to.

Lem0nTree commented 5 years ago

P.s happy halloweed man :D

davidteather commented 5 years ago

Thanks, for some reason it's snowing on Halloween for me.

tmichel124 commented 5 years ago

what did you do to get around the trending page loading?

davidteather commented 5 years ago 
chromeProfile = webdriver.ChromeOptions()
chromeProfile.add_argument("--disable-automation")
chromeProfile.add_experimental_option("excludeSwitches", ["enable-automation"])
chromeProfile.add_argument("--proxy-server={0}".format(proxy.proxy))
chromeProfile.add_experimental_option("mobileEmulation", {"deviceName": "Galaxy S5"})

if headless == True:
    chromeProfile.add_argument('headless')
driver = webdriver.Chrome(chrome_options=chromeProfile)

Instead of the firefox driver on lines 31 in tiktok.py I had to switch to chrome driver. I'll release a version that works, when I find a solution for the other functions not working.

tmichel124 commented 5 years ago

thanks so much! that will help in the meantime

tmichel124 commented 5 years ago

I was trying this code and it still looks like all the responses are status 200 when I look at the proxy har, can you confirm this is still working for you?

It seems to work correctly when I comment out the headless argument line, though!

davidteather commented 5 years ago

I didn't test it as a headless browser honestly, I'm just trying to get the API to work first. It might not work on headless right now.

Lem0nTree commented 5 years ago

I checked other tiktok api on github, neither one of them seems to work :(

davidteather commented 5 years ago

Yeah I’ve also been struggling to get any type of response. The most I’ve gotten is the top 2 from trending but that’s it.

Lem0nTree commented 5 years ago

So no hope for make them work in future? I was working on a project with those api since september :(

Lem0nTree commented 5 years ago

Btw you have done a really great job ,if you have a paypal or so i wih to contribute to your work!

davidteather commented 5 years ago

I hope to make it work in the future, however no guarantees. I have a solution in mind, but it might take a while and not be as fast as before.

Just added a sponsorship button to the repository. Here's the link anyways. Also thanks for even considering to donate. I never really thought this api would get many downloads or even someone who wanted to donate.

Lem0nTree commented 5 years ago

You made a great work, i learned a lot with your code! I will make the donation this week, let me know if you also have a LTC address

davidteather commented 5 years ago

Thanks man!

LTC: MCFSPnGDiS5x7GMYBgDrfzBHVuR6rDwFek

davidteather commented 5 years ago

I released a new PyPi version that supports the downloading video by url method.

Here is the release.

Still working on fixing the other methods in my free time.

Lem0nTree commented 5 years ago

Here a small gift for your amazing work 👍

Keep up the great work!

davidteather commented 5 years ago

@theparcel Thanks so much!

Parallaps commented 5 years ago

Are there any updates?

davidteather commented 5 years ago

Unfortunately not really. I’m attempting to use selenium to get the divisions of the tiktok videos but it seems like that’ll even be limited because of their protection against this stuff.

mjaensch commented 5 years ago

I also tried a bit to play around with selenium to get your API working again. It seems like the script https://www.tiktok.com/akam/11/1f5e5639 is responsible for detecting selenium. When you run it through a deobfuscator you can see a lot of different approaches to detect selenium: https://pastebin.com/r21Jjkxm Interestingly the script is only loaded when I open a tiktok page in selenium.

Lem0nTree commented 5 years ago

This fix work for other sites that detect selenium: https://intellipaat.com/community/5490/can-a-website-detect-when-you-are-using-selenium-with-chromedriver

https://stackoverflow.com/questions/33225947/can-a-website-detect-when-you-are-using-selenium-with-chromedriver

Lem0nTree commented 5 years ago

Updated to the latest version, and added the chromedriver stuff; but got this error: DevTools listening on ws://127.0.0.1:55758/devtools/browser/54bd65b9-db59-4209-855a-5e134c8fbba0 [1118/145356.352:INFO:CONSOLE(1)] "version:0.1.7", source: https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/_app.js (1) [1118/145356.611:INFO:CONSOLE(1)] "Mixed Content: The page at 'https://www.tiktok.com/trending?lang=en' was loaded over HTTPS, but requested an insecure image 'http://p16-tiktokcdn-com.akamaized.net/obj/tiktok-obj/1617014531310594.JPEG'. This content should also be served over HTTPS.", source: https://verify-sg.byteoversea.com/static/pc_slide.js (1) Found signature, continuing. Numbers of video to download: 10 [1118/145359.570:INFO:CONSOLE(0)] "The resource https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/home.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriateasvalue and it is preloaded intentionally.", source: (0) [1118/145359.574:INFO:CONSOLE(0)] "The resource https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/explore.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriateasvalue and it is preloaded intentionally.", source: (0) [1118/145359.574:INFO:CONSOLE(0)] "The resource https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/about.js was preloaded using link preload but not used within a few seconds from the window's load event. Please make sure it has an appropriateasvalue and it is preloaded intentionally.", source: (0) Hashtag to target: meme [1118/145440.214:INFO:CONSOLE(1)] "version:0.1.7", source: https://s16.tiktokcdn.com/tiktok/falcon/_next/static/1.0.1.157/pages/_app.js (1) [1118/145440.532:INFO:CONSOLE(0)] "Mixed Content: The page at 'https://www.tiktok.com/tag/memes?lang=en' was loaded over HTTPS, but requested an insecure image 'http://p16-tiktokcdn-com.akamaized.net/obj/tiktok-obj/1617014531310594.JPEG'. This content should also be served over HTTPS.", source: https://www.tiktok.com/tag/memes?lang=en (0)

davidteather commented 5 years ago

What code are you running?

Lem0nTree commented 5 years ago

Added you to my project on github, to show you the code :)

Lem0nTree commented 5 years ago

Still didn't find a solution, i looked around on the web for possible workaround but can't make it work again :(

davidteather commented 5 years ago

What chromedriver version are you using? I don't get that error on V78

dombrocarl commented 5 years ago

have you been able to make this api works?

I can pay you 1000 dollars if you make that works :)

davidteather commented 5 years ago

@dombrocarl

The api works right now. It's not the same as before TikTok changed their website. It does not work on a headless browser. I've been attempting to make it the same as before they changed their website, but I don't think I'll be able to make it exactly like that.

Feel free to ask any questions you have using the new slightly different API.

davidteather commented 4 years ago

I think I might have found a work around that works for some reason. I currently have the trending method working for this api, but not the other ones yet.

It gives the json response like it used to before TikTok changed their website.

davidteather commented 4 years ago

@theparcel @tmichel124 @Parallaps @mjaensch @dombrocarl

I have a fix for this API that will be released to PyPi in about half an hour. I'm just drafting the release notes and it should be up.

reteps commented 4 years ago

The issue popped back up.

My changes:

chromeProfile.add_argument('--ignore-certificate-errors')
chromeProfile.add_argument('--ignore-ssl-errors')
# later in trending
t = 0.5
if data["statusCode"] == 0:
    ...
elif data['statusCode'] == 200:
    t *= 1.5
    print(f'Denied. sleeping for {t} sec')
    time.sleep(t)
    print('now trying again')

Gives me 

...
now trying again
Denied. sleeping for 145.96 sec

It was working earlier, but suddenly stopped. @davidteather

davidteather commented 4 years ago

@reteps you might want to try having no delay. The only reason this works is through spamming the servers until it responds.

reteps commented 4 years ago

It works with no delay, but that is pretty sus

davidteather commented 4 years ago

@reteps yeah it’s an unofficial api and for some reason you need to spam their servers to get a request