Closed PATAPOsha closed 1 year ago
Hi there,
Check this example out.
That is super weird... I don't know how, but it works!
I tried to iterate over cursor
param and change different CHALLENGE_ID
. Results are good!
I don't understand how requesting same url with single diffrent "x-tt-params" header may cause different response... but it works.
Thanks!
However, I tried to follow the same logic with "search by keyword", but cannot pick up correct set of params.
TikTok always returns same response, ignoring what I've set for keyword
or cursor
param in const PARAMS
original request:
curl -H "Host: www.tiktok.com" -H "Cookie: tt_csrf_token=3IgZwwf8-vsMWUCL1_-hhW-W1YTT17MT_0vY; tt_chain_token=OSGVta6ATdtjBG+7ula69w==; bm_sz=2BC6ED25098AC799ED9D34CD7E32C35E~YAAQLlkVAqX0IcqFAQAA647TFxK8uvqO4rvIR9fUXdntxCLehbz0yCBYBQJV+dDkJHpOqMqEwREBfykh9RhiCyCzN6BJHaxoxZXwmIXMc+cEppAFHoEVuR/6lTAemzIysOGbzRVAsyVu3SxK831NKbdNIlXHY3DDHnK97LdhCp+48MuoWuvOL7VKvXkULPdVggiHuGJEUUziF/9E5sXChMHncDX/KlMU3UCG9C1edOmNpumW83HNP9fX+jzAhmiS+WDN3mhio4HkCQGAkZBKc/cg3W2YQl7VrVnPcItvRGyOQrc=~4339504~4539443; tiktok_webapp_theme=light; __tea_cache_tokens_1988={%22_type_%22:%22default%22%2C%22user_unique_id%22:%227195946673510548998%22%2C%22timestamp%22:1675436989307}; ak_bmsc=C374B6DAF6DED61C353AA0555C23A8FA~000000000000000000000000000000~YAAQLlkVAqr0IcqFAQAAfpPTFxLWgYeeQyE+qtk6P3zJbsExivFK7G4l6JhCU9RbJ2epiUTCPwuBP6OTebkYMjTwn9sWypdVj88TidJvzsx13BWeskiXTNYzyhFs9nJpexnOAcRXLb1shFbO/il1OG+6HquYo3LhHL8qsfMzmRcPgVZdqAY/4KloTaDjab8MdAUp1litODJafawJ5TrOMGTingZwI/cnbGKbOo7StgOnCkzW1HQ95jx+4F80hGxNeH1xgWQVhA5FS+mHx/bGo5sy3p042Qxpemn8V/DxPkr4ANjpe/PLSex+9yuOC37vOIOCNQrndhaivR3uX7b6SnIFhPPYiVs1hmlmGokN/aBn6ynXvtOdJXbly0q+1KnXZ8Tcf7gcdGD5/TYO/jTSxVeCAuc0adAaHPnna9q+XFxl4npeJBkDouyaXAvrs9uDySFhTTOFUX+IgXZF0jEN7vqo403I227l39PZpQ/29DFGw1tQhXkI8r5ttGXZiMU/zgcdN8U=; ttwid=1%7CjlDdDy6xwx_sx-68zmCBYY3mKTBkTajO7CF52fMhoYY%7C1675436987%7C25023915df27377f763e8d01e6727323a963687834caf05823e22573fe37f3c2; _abck=6571C0CBCCBDB25568BCB596C198E0F9~-1~YAAQLlkVArL0IcqFAQAAt6LTFwn+VN0RdUAA65gJR4cBUW66VMiNSwSO5bVeom/7PTe5kEdSyxGDq7R0qikPvt1SDeICA8vQhyNeAdHuWP6awbpYlGYfCeIbLwV4iW8mYcbSmVbmTZmRs6LvAuXre4XYNnIWAJnCdCCEI4c88rsawEhaQPtG2M42rvc1E0Ca/6QWaDZwtHlMWhbv6K7em7vpWVSzeBC1sSiUS5tBGZGf8WaFxaHDwDOeZwnsheA3HsAeVdnSYMGIS/AVx5C06REJDdjt0yNQnVKKHfc244yMlk9KLZuj/3S2m4cwwoI95/PkJ+oE7Rbb2xq6IwTDVy1cfngGOwZ3bF9hB3hPEqIIHSr19YSqS6YQQwznfTlzTjCq0qs/uxUgCw==~-1~-1~-1; msToken=Uq-Ry8Fb1qLo9-HNTrEzlfDXz_22Xx47iBqwXN40xV5y53aEIhXst249uxqi8kqbfu1mbZIS6y3hBePO-Pcpd8GdvMoy3exjMqupK478-_koinsmibMsepmXyoApV37mC2mlsEol6A7m; msToken=ryRSoItotPOQ6MPVjcyJyEDOqCa3ULMocYDV86C1mzYM8tNrvWU3MBS08pFQCi82I3fRAP5dcfNsWEb-KU00NIRs2dJJW1GbOFeqAO5swboRYvOPZJLDeE42oTS1yItVZOxciUtqJbiF" -H "sec-ch-ua: \"Not_A Brand\";v=\"99\", \"Google Chrome\";v=\"109\", \"Chromium\";v=\"109\"" -H "dnt: 1" -H "sec-ch-ua-mobile: ?0" -H "user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" -H "sec-ch-ua-platform: \"Windows\"" -H "accept: */*" -H "sec-fetch-site: same-origin" -H "sec-fetch-mode: cors" -H "sec-fetch-dest: empty" -H "referer: https://www.tiktok.com/search?q=beauty&t=1675437001724" -H "accept-language: en-US,en;q=0.9" --compressed "https://www.tiktok.com/api/search/general/full/?aid=1988&app_language=en&app_name=tiktok_web&battery_info=1&browser_language=en-US&browser_name=Mozilla&browser_online=true&browser_platform=Win32&browser_version=5.0%20%28Windows%20NT%2010.0%3B%20Win64%3B%20x64%29%20AppleWebKit%2F537.36%20%28KHTML%2C%20like%20Gecko%29%20Chrome%2F109.0.0.0%20Safari%2F537.36&channel=tiktok_web&cookie_enabled=true&device_id=7195946673510548998&device_platform=web_pc&focus_state=true&from_page=search&history_len=5&is_fullscreen=false&is_page_visible=true&keyword=beauty&offset=0&os=mac&priority_region=&referer=®ion=UA&screen_height=1440&screen_width=3440&tz_name=Europe%2FKiev&webcast_language=en&msToken=Uq-Ry8Fb1qLo9-HNTrEzlfDXz_22Xx47iBqwXN40xV5y53aEIhXst249uxqi8kqbfu1mbZIS6y3hBePO-Pcpd8GdvMoy3exjMqupK478-_koinsmibMsepmXyoApV37mC2mlsEol6A7m&X-Bogus=DFSzKwVLsbJANrqhS4qqTTXyYJW8&_signature=_02B4Z6wo00001WVSydAAAIDARvPnfx9Tb5llQs1AADqN0e"
I try to change keyword
and set "x-tt-params" header like you did in your example, but getting error from TikTok:
{
"status_code": 2483,
"status_msg": "Please login your account first",
"log_pb": {
"impr_id": "202302031511095FAA5ABFD2C11C1CAF2B"
}
}
And after I set cookies (copy from original request) - I get same response as for original request.
My code:
import requests
from urllib.parse import urlencode
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36"
TT_REQ_PERM_URL_SEARCH = "https://www.tiktok.com/api/search/general/full/?aid=1988&app_language=en&app_name=tiktok_web&battery_info=1&browser_language=en-US&browser_name=Mozilla&browser_online=true&browser_platform=Win32&browser_version=5.0%20%28Windows%20NT%2010.0%3B%20Win64%3B%20x64%29%20AppleWebKit%2F537.36%20%28KHTML%2C%20like%20Gecko%29%20Chrome%2F109.0.0.0%20Safari%2F537.36&channel=tiktok_web&cookie_enabled=true&device_id=7195946673510548998&device_platform=web_pc&focus_state=true&from_page=search&history_len=5&is_fullscreen=false&is_page_visible=true&keyword=beauty&offset=0&os=mac&priority_region=&referer=®ion=UA&screen_height=1440&screen_width=3440&tz_name=Europe%2FKiev&webcast_language=en&msToken=Uq-Ry8Fb1qLo9-HNTrEzlfDXz_22Xx47iBqwXN40xV5y53aEIhXst249uxqi8kqbfu1mbZIS6y3hBePO-Pcpd8GdvMoy3exjMqupK478-_koinsmibMsepmXyoApV37mC2mlsEol6A7m&X-Bogus=DFSzKwVLsbJANrqhS4qqTTXyYJW8&_signature=_02B4Z6wo00001WVSydAAAIDARvPnfx9Tb5llQs1AADqN0e"
headers = {
'sec-ch-ua': '"Not_A Brand";v="99", "Google Chrome";v="109", "Chromium";v="109"',
'dnt': '1',
'sec-ch-ua-mobile': '?0',
'user-agent': USER_AGENT,
'sec-ch-ua-platform': '"Windows"',
'accept': '*/*',
'origin': 'https://www.tiktok.com',
'sec-fetch-site': 'same-site',
'sec-fetch-mode': 'cors',
'sec-fetch-dest': 'empty',
'referer': 'https://www.tiktok.com/',
'accept-language': 'en-US,en;q=0.9',
}
search_cookies = {
'tt_csrf_token': '3IgZwwf8-vsMWUCL1_-hhW-W1YTT17MT_0vY',
'tt_chain_token': 'OSGVta6ATdtjBG+7ula69w==',
'bm_sz': '2BC6ED25098AC799ED9D34CD7E32C35E~YAAQLlkVAqX0IcqFAQAA647TFxK8uvqO4rvIR9fUXdntxCLehbz0yCBYBQJV+dDkJHpOqMqEwREBfykh9RhiCyCzN6BJHaxoxZXwmIXMc+cEppAFHoEVuR/6lTAemzIysOGbzRVAsyVu3SxK831NKbdNIlXHY3DDHnK97LdhCp+48MuoWuvOL7VKvXkULPdVggiHuGJEUUziF/9E5sXChMHncDX/KlMU3UCG9C1edOmNpumW83HNP9fX+jzAhmiS+WDN3mhio4HkCQGAkZBKc/cg3W2YQl7VrVnPcItvRGyOQrc=~4339504~4539443',
'tiktok_webapp_theme': 'light',
'__tea_cache_tokens_1988': '{%22_type_%22:%22default%22%2C%22user_unique_id%22:%227195946673510548998%22%2C%22timestamp%22:1675436989307}',
'ak_bmsc': 'C374B6DAF6DED61C353AA0555C23A8FA~000000000000000000000000000000~YAAQLlkVAqr0IcqFAQAAfpPTFxLWgYeeQyE+qtk6P3zJbsExivFK7G4l6JhCU9RbJ2epiUTCPwuBP6OTebkYMjTwn9sWypdVj88TidJvzsx13BWeskiXTNYzyhFs9nJpexnOAcRXLb1shFbO/il1OG+6HquYo3LhHL8qsfMzmRcPgVZdqAY/4KloTaDjab8MdAUp1litODJafawJ5TrOMGTingZwI/cnbGKbOo7StgOnCkzW1HQ95jx+4F80hGxNeH1xgWQVhA5FS+mHx/bGo5sy3p042Qxpemn8V/DxPkr4ANjpe/PLSex+9yuOC37vOIOCNQrndhaivR3uX7b6SnIFhPPYiVs1hmlmGokN/aBn6ynXvtOdJXbly0q+1KnXZ8Tcf7gcdGD5/TYO/jTSxVeCAuc0adAaHPnna9q+XFxl4npeJBkDouyaXAvrs9uDySFhTTOFUX+IgXZF0jEN7vqo403I227l39PZpQ/29DFGw1tQhXkI8r5ttGXZiMU/zgcdN8U=',
'ttwid': '1%7CjlDdDy6xwx_sx-68zmCBYY3mKTBkTajO7CF52fMhoYY%7C1675436987%7C25023915df27377f763e8d01e6727323a963687834caf05823e22573fe37f3c2',
'_abck': '6571C0CBCCBDB25568BCB596C198E0F9~-1~YAAQLlkVArL0IcqFAQAAt6LTFwn+VN0RdUAA65gJR4cBUW66VMiNSwSO5bVeom/7PTe5kEdSyxGDq7R0qikPvt1SDeICA8vQhyNeAdHuWP6awbpYlGYfCeIbLwV4iW8mYcbSmVbmTZmRs6LvAuXre4XYNnIWAJnCdCCEI4c88rsawEhaQPtG2M42rvc1E0Ca/6QWaDZwtHlMWhbv6K7em7vpWVSzeBC1sSiUS5tBGZGf8WaFxaHDwDOeZwnsheA3HsAeVdnSYMGIS/AVx5C06REJDdjt0yNQnVKKHfc244yMlk9KLZuj/3S2m4cwwoI95/PkJ+oE7Rbb2xq6IwTDVy1cfngGOwZ3bF9hB3hPEqIIHSr19YSqS6YQQwznfTlzTjCq0qs/uxUgCw==~-1~-1~-1',
'msToken': 'Uq-Ry8Fb1qLo9-HNTrEzlfDXz_22Xx47iBqwXN40xV5y53aEIhXst249uxqi8kqbfu1mbZIS6y3hBePO-Pcpd8GdvMoy3exjMqupK478-_koinsmibMsepmXyoApV37mC2mlsEol6A7m',
'msToken': 'ryRSoItotPOQ6MPVjcyJyEDOqCa3ULMocYDV86C1mzYM8tNrvWU3MBS08pFQCi82I3fRAP5dcfNsWEb-KU00NIRs2dJJW1GbOFeqAO5swboRYvOPZJLDeE42oTS1yItVZOxciUtqJbiF',
}
sess = requests.session()
sess.headers.update(headers)
def request_search_list(keyword: str, count: int = 12, offset: int = 0):
params = {
"aid": "1988",
# "count": count,
"keyword": keyword,
"offset": offset,
"cookie_enabled": True,
"screen_width": 0,
"screen_height": 0,
"browser_language": "",
"browser_platform": "",
"browser_name": "",
"browser_version": "",
"browser_online": "",
"timezone_name": "Europe/London",
# "priority_region": "US",
# "region": "US",
# "language": "en",
# "webcast_language": "en",
}
unsigned_url = 'https://www.tiktok.com/api/search/general/full/?' + urlencode(params, quote_via=urllib.parse.quote)
signature_data = request_signature(unsigned_url)
ts = int(time.time() * 10**3)
referer = f"https://www.tiktok.com/search?q={keyword}&t={ts}"
headers.update({
"user-agent": signature_data['data']['navigator']['user_agent'],
"x-tt-params": signature_data['data']['x-tt-params'],
"referer": referer
})
r = sess.get(TT_REQ_PERM_URL_SEARCH, headers=headers, verify=False, cookies=search_cookies)
print(r.text)
def request_signature(tiktok_url: str):
url = "http://127.0.0.1/signature"
r = requests.post(url, data=tiktok_url, headers={'Content-type': 'application/json'})
json_resp = r.json()
return json_resp
if __name__ == '__main__':
posts = request_search_list("mustang")
How do you pick up correct set of params?
Can confirm that this method also works for https://www.tiktok.com/api/user/detail/?
and https://www.tiktok.com/api/related/item_list/?
, bot not for "search by keyword".
@carcabot i have same question with PATAPOsha And can we make a post request to tiktok with x-tt-params I see that X-Bogus generation function use raw post body request as 2nd argument so i think it is not much differrent Wait for your reply
I mean that it is good if we can make a comment on tiktok live chat using this project @carcabot
hey @PATAPOsha, does the user-videos.js example work for you? it seems like the static URL doesn't work anymore.
ablago-mw Yes, it worked for me.
I copied my own fresh TT_REQ_PERM_URL_USER_POSTS from real browser, and followed same logic like in user-videos.js
:
TT_REQ_PERM_URL_USER_POSTS = "https://us.tiktok.com/api/post/item_list/?aid=1988&app_language=en&app_name=tiktok_web&battery_info=1&browser_language=en-US&browser_name=Mozilla&browser_online=true&browser_platform=Win32&browser_version=5.0%20%28Windows%20NT%2010.0%3B%20Win64%3B%20x64%29%20AppleWebKit%2F537.36%20%28KHTML%2C%20like%20Gecko%29%20Chrome%2F109.0.0.0%20Safari%2F537.36&channel=tiktok_web&cookie_enabled=true&device_id=7195955847507740166&device_platform=web_pc&focus_state=true&from_page=user&history_len=7&is_fullscreen=false&is_page_visible=true&os=mac&priority_region=&referer=®ion=US&screen_height=1440&screen_width=3440&tz_name=Europe%2FKiev&webcast_language=en&msToken=6JxScgMyaRPESPCETDAsgvTwQREUZ76t37atjrtzaBm2-W-_N6Pll3jYXsHFUJAX-jyNmQkiJYX9QdcxvxPi2dRZwxNe2WbcPetdg0HgHG3kHTayGQUMDLBLXOmxv1_pR--X5MjI_4he&X-Bogus=DFSzKwVYqZUANaWzS4mVHTXyYJlw&_signature=_02B4Z6wo00001cbaXCgAAIDA5XtyhpreW9nGyliAABJrb4"
def request_user_posts(sec_user_id: str, cursor: int = 0):
params = {
"aid": "1988",
"count": 30,
"secUid": sec_user_id,
"cursor": cursor,
"cookie_enabled": True,
"screen_width": 0,
"screen_height": 0,
"browser_language": "",
"browser_platform": "",
"browser_name": "",
"browser_version": "",
"browser_online": "",
"timezone_name": "Europe/London",
"priority_region": "US",
"region": "US",
"language": "en",
"webcast_language": "en",
}
unsigned_url = 'https://us.tiktok.com/api/post/item_list/?' + urlencode(params, quote_via=urllib.parse.quote)
signature_data = request_signature(unsigned_url)
headers.update({
"user-agent": signature_data['data']['navigator']['user_agent'],
"x-tt-params": signature_data['data']['x-tt-params'],
})
r = sess.get(TT_REQ_PERM_URL_USER_POSTS, headers=headers, verify=False)
return
if __name__ == '__main__':
# user_sec_id = "MS4wLjABAAAAlH8McDkHdG1TFTPEFeCwIivQdwDFVmJsTONZDPYUJFnW4mRN-6BVwOmIED0TOvX3" # @yourhomeandmine
user_sec_id = "MS4wLjABAAAAgswV4bnE3b_fY_mhPPI0aW8-ugDVVTWhYyrci3TJuZkBpjXlg-yBwsvBF8r4pg-I" # @manisehgal
request_user_posts(user_sec_id)
And different secUid
returned me different valid responses.
Thanks! It looks like they burnt the user_agent from the example, using another user_agent fixed it. I hope it will last :P
@PATAPOsha @ablago-mw hi, is it still working for you guys ?
@PATAPOsha @ablago-mw hi, is it still working for you guys ?
yes
@PATAPOsha Oh nice!
What are you using for TT_REQ_PERM_URL_USER_POSTS and USER_AGENT please? tried everything and can't make it work :(
Thank you for your help <3
@Spawnrad just tested my old script above with fresh USER_AGENT
and TT_REQ_PERM_URL_USER_POSTS
. Everything works as before.
I used
USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36"
TT_REQ_PERM_URL_USER_POSTS = "https://us.tiktok.com/api/post/item_list/?aid=1988&app_language=en&app_name=tiktok_web&battery_info=1&browser_language=en-US&browser_name=Mozilla&browser_online=true&browser_platform=Win32&browser_version=5.0%20%28Windows%20NT%2010.0%3B%20Win64%3B%20x64%29%20AppleWebKit%2F537.36%20%28KHTML%2C%20like%20Gecko%29%20Chrome%2F111.0.0.0%20Safari%2F537.36&channel=tiktok_web&cookie_enabled=true&device_id=7213360202673194538&device_platform=web_pc&focus_state=true&from_page=user&history_len=5&is_fullscreen=false&is_page_visible=true&os=windows&priority_region=&referer=®ion=US&screen_height=1440&screen_width=3440&tz_name=Europe%2FKiev&webcast_language=en&msToken=4qGYpz_1jUFPOH7BMO2G_CovRg_fRdfC72-NGoz24_BaLw6QcTOYymGF8OLatoajpd3JRI7u3frt2cuyTdzW-BRbyZQCM3H5XXtGnYLpPdtpexK_lMny&X-Bogus=DFSzswVusO2ANaR0tce472XyYJUn&_signature=_02B4Z6wo00001Xh2bjAAAIDAW9dAn85oCCV4dmqAADoKfe"
You better copy your own donor TT_REQ_PERM_URL_USER_POSTS
from chrome dev-tools/Charles/Burp.
Make sure you put same USER_AGENT
here https://github.com/carcabot/tiktok-signature/blob/master/index.js#L7 !
Describe the bug I try to get list of posts by hashtag using
https://us.tiktok.com/api/challenge/item_list/?
, but after puttingX-Bogus
+_signature
generated with this project - I get empty response (200 status code).Final failed request (with signature generated by
[tiktok-signature](https://github.com/carcabot/tiktok-signature)
):Original good request (copied from browser):
I noticed that this project is based on
webmssdk/1.0.0.211
, however official tiktok web-app useswebmssdk/1.0.0.12
for me:https://sf16-website-login.neutral.ttwstatic.com/obj/tiktok_web_login_static/webmssdk/1.0.0.12/webmssdk.js
Also, your signature object also includes
x-tt-params
andverify_fp
which are not included in original requests today. But nothing said aboutmsToken
param, which seems to be required.Seems like this project's webmssdk version is outdated. If so, can anybody share old working urls for searching by hashtag, keyword, getting userpofile info and post details?
Also, I will be grateful if somebody points me a method that generates
X-Bogus
. All I found isfunction _0x171e0b(_0x5652a1, _0x2d0381, _0x489f15, _0x360203, _0x37c283)
, that addsX-Bogus
,_signature
andmsToken
to the existingRequest
object. But it also triggers an actual request, which is not needed.Also, I noticed that
_signature
param is not required. Request copied from browser works without_signature
param, onlymsToken
andX-Bogus
are required.msToken
is updated with cookie with every request, butX-Bogus
is a problem.If fields below will be empty then issue will be Ignored and Closed
To Reproduce Steps to reproduce the behavior:
Screenshots If applicable, add screenshots to help explain your problem.