davidteather / TikTok-Api

The Unofficial TikTok API Wrapper In Python
https://davidteather.github.io/TikTok-Api
MIT License
4.84k stars 973 forks source link

[BUG] - Code doesn't return anything. #891

Closed FlashAndromeda closed 1 year ago

FlashAndromeda commented 2 years ago

I don't get any output from any code, even the examples provided with the package and in the readme

import logging
from TikTokApi import TikTokApi

with TikTokApi(logging_level=logging.INFO) as api:
    for trending_video in api.trending.videos(count=50):
        print(trending_video.author.username)

I've tried reinstalling it, installing it from outside the venv, reinstalling playwright, passing the cookie to the API and nothing changed. Is anyone else having the same issue?

robindz commented 2 years ago

Can confirm, same issue

FlashAndromeda commented 2 years ago

I have managed to get it to work simply by manually opening tiktok in the browser before running the script on the machine. Could it maybe be due to anti-bot measures on tiktoks side and no/faulty error handling on the packages side?

thaornguyen commented 2 years ago

Same issue, i can't get api.user(). Error is: "TikTok blocks this request displaying a Captcha". I used proxy and changed custom_verify_fp but not solved.

adamd01 commented 2 years ago

I was having this issue, but I noticed the url generated worked fine in my browser

So I exported the cookies from my browser (I'm logged into TikTok) into cookies.json and then was able to use the below to set those cookies in all requests through the api

import json

def get_cookies_from_file():
    with open('cookies.json') as f:
        cookies = json.load(f)

    cookies_kv = {}
    for cookie in cookies:
        cookies_kv[cookie['name']] = cookie['value']

    return cookies_kv

cookies = get_cookies_from_file()

def get_cookies(**kwargs):
    return cookies

api = TikTokApi()

api._get_cookies = get_cookies  # This fixes issues the api was having

This is working for me

dhudsmith commented 2 years ago

UPDATE: I was able to get @adam01's solution to work. I was able to get the cookies.json in the correct form by using Chrome along with the Cookie Editor extension (https://add0n.com/cookie-editor.html)

@adamd01, thanks for sharing this. ~I can't get it to work.~ Can you give a little more detail about which cookies file you saved? I'm seeing 33 cookies for the tiktok domain. I exported all of these as a cookies.json. I get an array of the following objects. Where are the "key" and "value" coming from in your snippet.

{
    "Host raw": "https://.tiktok.com/",
    "Name raw": "tt_csrf_token",
    "Path raw": "/",
    "Content raw": "<redacted>",
    "Expires": "At the end of the session",
    "Expires raw": "0",
    "Send for": "Encrypted connections only",
    "Send for raw": "true",
    "HTTP only raw": "true",
    "SameSite raw": "lax",
    "This domain only": "Valid for subdomains",
    "This domain only raw": "false",
    "Store raw": "firefox-default",
    "First Party Domain": ""
}

FYI, I'm on firefox and I am using the "Quick Cookie Manager" to export the json.

Wathfea commented 2 years ago

@dhudsmith or @adamd01 can you please describe step by step what did you do? I tried to recreate what you sad but I'm out of luck.

I saved the cookie.json in the above format what you wrote and chaged the code for the correct key value pairs bt the script still idle and I can't get back any response.

Here is my code:

import json

def get_cookies_from_file():
    with open('/Users/davidperlusz/code/tiktok-scraper/cookies.json') as f:
        cookies = json.load(f)

    cookies_kv = {}
    for cookie in cookies:
        cookies_kv[cookie['Name raw']] = cookie['Content raw"']

    return cookies_kv

cookies = get_cookies_from_file()

def get_cookies(**kwargs):
    return cookies

from TikTokApi import TikTokApi

api = TikTokApi()

api._get_cookies = get_cookies  # This fixes issues the api was having

for trending_video in api.trending.videos(count=50):
    print(trending_video.author.username)
adamd01 commented 2 years ago

@Wathfea - I'm using "edit this cookie" chrome plugin to do the export of cookies

My exported file looks like this:

[
{
    "domain": ".tiktok.com",
    "expirationDate": 1686154900,
    "hostOnly": False,
    "httpOnly": False,
    "name": "_abck",
    "path": "/",
    "sameSite": "unspecified",
    "secure": True,
    "session": False,
    "storeId": "0",
    "value": "XXX",
    "id": 1
},
...(above format repeated for each cookie)

Although the only thing we need is the name and value fields, as long as those are populated the code will work And in my code you can see I actually had to format it to be a more simple format before passing it through, so eventually you just have

cookies = {'name_of_cookie': 'value', ...}
dhudsmith commented 2 years ago

Here's the step-by-step that worked for me, @Wathfea:

  1. Install this cookie manager extension for Chrome (others may work): https://chrome.google.com/webstore/detail/cookiemanager-cookie-edit/
  2. Log into TikTok using Chrome
  3. Using the cookie manager extension, view all cookies from tiktok.
  4. Select and export all as a cookies.json file. These will have the keys "name" and "value".
  5. Use the code provided by @adamd01
Wathfea commented 2 years ago

Thank you guys it is working now!

lazezo2 commented 2 years ago

api.trending -- works only api.user -- not working api.sound -- not working

janith-jware commented 2 years ago

api.trending -- works only api.user -- not working api.sound -- not working

Yes

janith-jware commented 2 years ago

api.user is also working when you pass the output of trending_video.author to user as follows:

user = trending_video.author
print(user.info_full())

This works!!

lazezo2 commented 2 years ago

any idea how to get user feed and sound feed? using this method to add cookies make api.trending to work!, but all other function not working....

thaornguyen commented 2 years ago

@Wathfea - I'm using "edit this cookie" chrome plugin to do the export of cookies

My exported file looks like this:

[
{
    "domain": ".tiktok.com",
    "expirationDate": 1686154900,
    "hostOnly": False,
    "httpOnly": False,
    "name": "_abck",
    "path": "/",
    "sameSite": "unspecified",
    "secure": True,
    "session": False,
    "storeId": "0",
    "value": "XXX",
    "id": 1
},
...(above format repeated for each cookie)

Although the only thing we need is the name and value fields, as long as those are populated the code will work And in my code you can see I actually had to format it to be a more simple format before passing it through, so eventually you just have

cookies = {'name_of_cookie': 'value', ...}

this idea works till today, Is your code still working?

DominicInitialing commented 2 years ago

Using the cookies exported by manual are working! But I wanted to get the cookies directly by script, so I tried below:

def get_cookies_from_website():
    sessions = requests.Session().get(r'https://www.tiktok.com')
    return session.cookies.get_dict()

I got just part of the cookies compared to the worked one. How could I fix it?

thaornguyen commented 2 years ago

I used cookies and returned userinfo data (r.text) as follows: https://gist.githubusercontent.com/thaornguyen/15de7065cb5091cfb25f4bd13222393a/raw/5f5f0e92b1b4ea899a9c72714886560af1898b23/gistfile1.txt

.......

How to get the data in the script tag?

Wathfea commented 2 years ago

@Wathfea - I'm using "edit this cookie" chrome plugin to do the export of cookies My exported file looks like this:

[
{
    "domain": ".tiktok.com",
    "expirationDate": 1686154900,
    "hostOnly": False,
    "httpOnly": False,
    "name": "_abck",
    "path": "/",
    "sameSite": "unspecified",
    "secure": True,
    "session": False,
    "storeId": "0",
    "value": "XXX",
    "id": 1
},
...(above format repeated for each cookie)

Although the only thing we need is the name and value fields, as long as those are populated the code will work And in my code you can see I actually had to format it to be a more simple format before passing it through, so eventually you just have

cookies = {'name_of_cookie': 'value', ...}

this idea works till today, Is your code still working?

Yes it's working for me too, but I not used it anymore because it not fits my needs :) Thanks anyway!

adnan-abbas commented 2 years ago

@dhudsmith or @adamd01 can you please describe step by step what did you do? I tried to recreate what you sad but I'm out of luck.

I saved the cookie.json in the above format what you wrote and chaged the code for the correct key value pairs bt the script still idle and I can't get back any response.

Here is my code:

import json

def get_cookies_from_file():
    with open('/Users/davidperlusz/code/tiktok-scraper/cookies.json') as f:
        cookies = json.load(f)

    cookies_kv = {}
    for cookie in cookies:
        cookies_kv[cookie['Name raw']] = cookie['Content raw"']

    return cookies_kv

cookies = get_cookies_from_file()

def get_cookies(**kwargs):
    return cookies

from TikTokApi import TikTokApi

api = TikTokApi()

api._get_cookies = get_cookies  # This fixes issues the api was having

for trending_video in api.trending.videos(count=50):
    print(trending_video.author.username)

What is Name raw and Content raw in @adamd01 code?

adamd01 commented 2 years ago

@dhudsmith or @adamd01 can you please describe step by step what did you do? I tried to recreate what you sad but I'm out of luck. I saved the cookie.json in the above format what you wrote and chaged the code for the correct key value pairs bt the script still idle and I can't get back any response. Here is my code:

import json

def get_cookies_from_file():
    with open('/Users/davidperlusz/code/tiktok-scraper/cookies.json') as f:
        cookies = json.load(f)

    cookies_kv = {}
    for cookie in cookies:
        cookies_kv[cookie['Name raw']] = cookie['Content raw"']

    return cookies_kv

cookies = get_cookies_from_file()

def get_cookies(**kwargs):
    return cookies

from TikTokApi import TikTokApi

api = TikTokApi()

api._get_cookies = get_cookies  # This fixes issues the api was having

for trending_video in api.trending.videos(count=50):
    print(trending_video.author.username)

What is Name raw and Content raw in @adamd01 code?

It's a specific reference to the labels used in your cookies.json file, which will depend on how you exported the cookies.

I used editthiscookie in chrome, and for me the labels were simply name and value. So basically have a look in the json and use the appropriate key's for the name of the cookie and the actual cookie value respectively

vladimir-cloudypro commented 1 year ago

@adamd01 Where did you paste this code?

adamd01 commented 1 year ago

@adamd01 Where did you paste this code?

@vladimir-cloudypro assuming you mean code from this comment: https://github.com/davidteather/TikTok-Api/issues/891#issuecomment-1148930488

This goes at the start of your scraping script, and then you can use the api object as per your needs

chumbucketleadcook commented 1 year ago

@adamd01 this worked for me. THanks!

reece-barker commented 1 year ago

hi this is still not working for me despite everything and all the steps i get this error:

Traceback (most recent call last): File "/Users/reece_barker/GitHub/RustNite TikTok/run.py", line 26, in <module> for video in api.trending.videos(): File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/TikTokApi/api/trending.py", line 59, in videos res = Trending.parent.get_data(path, ttwid=ttwid, **kwargs) File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/TikTokApi/tiktok.py", line 266, in get_data ) = asyncio.get_event_loop().run_until_complete( File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete return future.result() File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/TikTokApi/browser_utilities/browser.py", line 212, in sign_url evaluatedPage = await page.evaluate( File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/playwright/async_api/_generated.py", line 8658, in evaluate await self._impl_obj.evaluate( File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/playwright/_impl/_page.py", line 395, in evaluate return await self._main_frame.evaluate(expression, arg) File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/playwright/_impl/_frame.py", line 277, in evaluate await self._channel.send( File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 61, in send return await self._connection.wrap_api_call( File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 482, in wrap_api_call return await cb() File "/Users/reece_barker/.pyenv/versions/3.10.3/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 97, in inner_send result = next(iter(done)).result() playwright._impl._api_types.Error: TypeError: undefined is not an object (evaluating 'S[A][m(a[oprand[1]],oprand[1])]')

It is important to note I am running this on a MacOS machine, could this be the reason I am having issues?

vladimir-mawla commented 1 year ago

@reece-barker try to downgrade playwright to 1.30.0 This should solve your issue

faizaanqureshi commented 1 year ago

I can now download data from videos, however I cannot download bytes from videos to download them.

Here is my code:

from TikTokApi import TikTokApi
import json

def get_cookies_from_file():
    with open('cookies.json') as f:
        cookies = json.load(f)

    cookies_kv = {}
    for cookie in cookies:
        cookies_kv[cookie['name']] = cookie['value']

    return cookies_kv

cookies = get_cookies_from_file()

def get_cookies(**kwargs):
    return cookies

api = TikTokApi()

api._get_cookies = get_cookies  # This fixes issues the api was having

# Watch https://www.youtube.com/watch?v=-uCt1x8kINQ for a brief setup tutorial
#videos = []

for trending_video in api.trending.videos(1):
    # Prints the author's username of the trending video.
    #videos.append(trending_video)

    video_data = trending_video.bytes()

    with open("test.mp4", mode="wb") as out_file:
        out_file.write(video_data)

It simply outputs a file that is 1 KB and unreadable. When printing the bytes to the console, it returns something about not having access.

wp07e commented 1 year ago

I'm getting the same thing as @faizaanqureshi when trying to download a video

<HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>

You don't have permission to access "http&#58;&#47;&#47;v16&#45;webapp&#45;prime&#46;us&#46;tiktok&#46;com&#47;video&#47;tos&#47;useast2a&#47;tos&#45;useast2a&#45;ve&#45;0068c001&#47;oklD8MWIIEHISC8Ce5RnbCHjfsAeQIlgbsOIEF&#47;&#63;" on this server.<P>
Reference&#32;&#35;18&#46;1c1a32b8&#46;1687483729&#46;1d1135d4
</BODY>
</HTML>
ohchad commented 1 year ago

Not working for me either, but I suggest the following cookie shortcut over get_cookies_from_file

#  rawdata = document.cookies
rawdata = '_ttp=2N1PnWeqGTiw...'
cookies = (dict(i.split('=', 1) for i in rawdata.split('; ')))
davidteather commented 1 year ago

should be fixed V6