[BUG] - Getting tiktoks by username fails by "KeyError: "name='csrf_session_id', domain=None, path=None"

[Incher25]

Hi, I try to get tiktoks info by username, it worked fine until about August 13, then it started to fail by captcha. I solved captcha issue by using proxy, but by_username() method started to fail by "KeyError: "name='csrf_session_id', domain=None, path=None". I'm using TikTokApi v4.0.1 and here is my code:

import sys
import json
import logging
from TikTokApi import TikTokApi

use_test_endpoints=False
username = sys.argv[1]
count = int(sys.argv[2])
proxy="http://ххххх"
requests_extra_kwargs={"verify": '/path/to/ca.crt'}

if len(sys.argv) > 3:
    custom_verifyFp=sys.argv[3]
else:
    custom_verifyFp=""

api = TikTokApi.get_instance(use_test_endpoints=use_test_endpoints, custom_verifyFp=custom_verifyFp, proxy=proxy, logging_level=logging.DEBUG, requests_extra_kwargs=requests_extra_kwargs)

tiktoks = json.dumps(api.by_username(username=username, count=count))

print(tiktoks)

And here is a response I get:

INFO:root:Class initalized
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): tiktok.com:443
DEBUG:urllib3.connectionpool:https://tiktok.com:443 "GET /@znp08?lang=en HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): m.tiktok.com:443
DEBUG:urllib3.connectionpool:https://m.tiktok.com:443 "HEAD /api/post/item_list/?aid=1988&app_name=tiktok_web&device_platform=web_mobile&region=US&priority_region=&os=ios&referer=&root_referer=&cookie_enabled=true&screen_width=1380&screen_height=600&browser_language=en-us&browser_platform=iPhone&browser_name=Mozilla&browser_version=Mozilla%252F5.0%2B%28iPhone%253B%2BCPU%2BiPhone%2BOS%2B12_2%2Blike%2BMac%2BOS%2BX%29%2BAppleWebKit%252F605.1.15%2B%28KHTML%2C%2Blike%2BGecko%29%2BVersion%252F13.0%2BMobile%252F15E148%2BSafari%252F604.1&browser_online=true&timezone_name=Asia%252FVladivostok&is_page_visible=true&focus_state=true&is_fullscreen=false&history_len=4&language=en&count=3&id=6747250096503129093&cursor=0&type=1&secUid=MS4wLjABAAAAhOwyBs6Q6m5ASaPVv4eeb2d_-2oZmrOe_OW-EPEUL-zhnexHCyZpXgO5Lb4_oRVr&sourceType=8&appId=1233&region=US&priority_region=US&language=en&verifyFp=my_verify_fp&device_id=2412217525666279038&_signature=_02B4Z6wo00f01mRCymQAAIBAW-pUEdFm1GJkU87AAPhE5a HTTP/1.1" 503 0
Traceback (most recent call last):
  File "/var/www/app/python/scripts/TikTok/get_tiktoks_by_username.py", line 20, in <module>
    tiktoks = json.dumps(api.by_username(username=username, count=count))
  File "/usr/local/lib/python3.7/dist-packages/TikTokApi/tiktok.py", line 622, in by_username
    **kwargs,
  File "/usr/local/lib/python3.7/dist-packages/TikTokApi/tiktok.py", line 582, in user_posts
    res = self.get_data(url=api_url, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/TikTokApi/tiktok.py", line 254, in get_data
    csrf_session_id = h.cookies["csrf_session_id"]
  File "/usr/local/lib/python3.7/dist-packages/requests/cookies.py", line 328, in __getitem__
    return self._find_no_duplicates(name)
  File "/usr/local/lib/python3.7/dist-packages/requests/cookies.py", line 399, in _find_no_duplicates
    raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
KeyError: "name='csrf_session_id', domain=None, path=None"

Also I've tried to use different custom_verifyFp but it makes no sense.

Any help would be appreciated, thanks in advance!

[returnWOW]

In my case, deleted parameter use_test_endpoints=use_test_endpoints solve this problem. maybe this api is disable.

[mjblski]

The parameter custom_verifyFp does not exist any more due tiktok had changed auth system - relogin into ttok and check cookies - there is no s_v_web_id. check #724

[adiantek]

I had this error too, added debugs and:

 INFO:root:send https://m.tiktok.com/api/post/item_list/?aid=1988&app_name=tiktok_web&device_platform=web_mobile&region=US&priority_region=&os=ios&referer=&root_referer=&cookie_enabled=true&screen_width=1043&screen_height=824&browser_language=en-us&browser_platform=iPhone&browser_name=Mozilla&browser_version=Mozilla%252F5.0%2B%28iPhone%253B%2BCPU%2BiPhone%2BOS%2B12_2%2Blike%2BMac%2BOS%2BX%29%2BAppleWebKit%252F605.1.15%2B%28KHTML%2C%2Blike%2BGecko%29%2BVersion%252F15.4%2BMobile%252F15E148%2BSafari%252F604.1&browser_online=true&timezone_name=UTC&is_page_visible=true&focus_state=true&is_fullscreen=false&history_len=17&language=en&count=30&id=6780984514191393798&type=1&secUid=MS4wLjABAAAAZ_O95oMQSqpfH_9djQHrARdrRJOgZIH3dK-EouGardgvNcAVGtzzMPom2KYbBg2m&cursor=1629721887000&sourceType=8&appId=1233&region=US&language=en
 INFO:root:head URL: https://m.tiktok.com/api/post/item_list/?aid=1988&app_name=tiktok_web&device_platform=web_mobile&region=US&priority_region=&os=ios&referer=&root_referer=&cookie_enabled=true&screen_width=1043&screen_height=824&browser_language=en-us&browser_platform=iPhone&browser_name=Mozilla&browser_version=Mozilla%252F5.0%2B%28iPhone%253B%2BCPU%2BiPhone%2BOS%2B12_2%2Blike%2BMac%2BOS%2BX%29%2BAppleWebKit%252F605.1.15%2B%28KHTML%2C%2Blike%2BGecko%29%2BVersion%252F15.4%2BMobile%252F15E148%2BSafari%252F604.1&browser_online=true&timezone_name=UTC&is_page_visible=true&focus_state=true&is_fullscreen=false&history_len=17&language=en&count=30&id=6780984514191393798&type=1&secUid=MS4wLjABAAAAZ_O95oMQSqpfH_9djQHrARdrRJOgZIH3dK-EouGardgvNcAVGtzzMPom2KYbBg2m&cursor=1629721887000&sourceType=8&appId=1233&region=US&language=en&verifyFp=verify_33976c781f84a13a21356652d0ac89df&device_id=1533093112124503653&_signature=_02B4Z6wo00f01XjCeDgAAIBDR2rmTIsLDVV403yAAD9S75
 INFO:root:response: 404 / {'Server': 'nginx', 'Content-Type': 'text/plain; charset=utf-8', 'Content-Length': '23', 'X-Janus-Mini-Api-Forward': 'Janus-Mini(fast)', 'Content-Encoding': 'gzip', 'X-Tt-Logid': '20211031173700010190176040522690B8', 'Strict-Transport-Security': 'max-age=31536000', 'Server-Timing': 'inner; dur=40, cdn-cache; desc=MISS, edge; dur=0, origin; dur=403', 'x-tt-trace-host': '01d027ded10c2c3736a86e0fc253ce29da097c81f4d9a70a87a9a6af4e4d02af7720ac92852aaf78d8c0e9b51a2038d9ede2585afbd6caf5f438de1c7fd849b4d8c2c033a1228426378677a59e6e53a9dbeb85c1eb61e58da165370be0a471debe', 'Access-Control-Expose-Headers': 'X-Tt-Logid', 'Date': 'Sun, 31 Oct 2021 17:37:00 GMT', 'Connection': 'keep-alive', 'Vary': 'Accept-Encoding', 'x-tt-trace-tag': 'id=16;cdn-cache=miss;type=dyn', 'X-Origin-Response-Time': '403,23.213.160.86'}
 ----------------------------------------
 Traceback (most recent call last):
     resp = api.user_page("...", "...", page_size=30, cursor=cursor)
   File "/TikTokApi/TikTokApi/tiktok.py", line 691, in user_page
     return self.get_data(url=api_url, send_tt_params=True, **kwargs)
   File "/TikTokApi/TikTokApi/tiktok.py", line 272, in get_data
     csrf_session_id = h.cookies["csrf_session_id"]
   File "/usr/local/lib/python3.8/dist-packages/requests/cookies.py", line 328, in __getitem__
     return self._find_no_duplicates(name)
   File "/usr/local/lib/python3.8/dist-packages/requests/cookies.py", line 399, in _find_no_duplicates
     raise KeyError('name=%r, domain=%r, path=%r' % (name, domain, path))
 KeyError: "name='csrf_session_id', domain=None, path=None"
 ----------------------------------------

I had exactly 3 such errors when I called x2363 user_posts (each was valid uid, secUID, always page_size=30). PS. It looks like x-secsdk-csrf-token is an optional header. I removed it and it works for me, but I'm using only JSON API (only user_page, without any get_user_object, I cached uid and secUID). it sent a random string earlier: https://github.com/davidteather/TikTok-Api/commit/e70849477f5e27b4760d0a5d610834bdeeb60f6d

[moseoridev]

Removing all csrf_session_id things in tiktok.py will solve this issue. But I'm not sure it's right solution 😔

[davidteather]

I was getting this while testing out V4.1.0 on the by_username and by_hashtag endpoints for a bit. I'm going to self-assign myself. I think it won't happen as much as before with V4.1.0 but sometimes still happens. Think it has to do with getting detected making an HTTP call to tiktok to generate the csrf_session_id

Proxies might help, but they'll probably have to be pretty good

[davidteather]

Please try using V5.0.0 this may solve your problem

[Jheesbrough]

Still having this problem with version V5.0.0 after calling maybe ten things in a row.

[davidteather]

it's because they're detecting you're making too many requests in a row

[petarpetarpetar]

Any solution to this?

[xaviorlewis]

The max number of requests I seem to be able to make is 13 - are there any updates on a solution to this? I tried changing the did variable every so often, but that had no effect. Would waiting longer between requests be helpful?