avilash / TikTokAPI-Python

TikTok API Python Wrapper
http://avilashkumar.com/TikTokAPI-Python/
MIT License
445 stars 113 forks source link

JSONDecodeError #3

Closed ACLzz closed 4 years ago

ACLzz commented 4 years ago

Hello! I'm trying to do this:

class Bot:
    t: TikTokAPI
    hashtags = [
        'usatiktokers',
        'washington'
    ]

    def __init__(self):
        log.info("Initializing bot...")
        self.t = TikTokAPI(region="US")
        log.info("Initialized.")

    def crawl(self, count: int) -> list:
        log.info(f"Starting crawling for {count} videos...")
        result = []
        count_per_seed = int(count / len(self.hashtags))

        for hashtag in self.hashtags:
            result.extend(self.t.getVideosByHashTag(hashTag=hashtag, count=count_per_seed)['body']['itemListData'])

But i got this:

2020-08-18 00:05:49,348 Initializing bot...
2020-08-18 00:05:49,348 Initialized.
2020-08-18 00:05:49,348 Starting crawling for 1000 videos...
Traceback (most recent call last):
  File "bot.py", line 78, in <module>
    videos = t.crawl(1000)
  File "bot.py", line 42, in crawl
    result.extend(self.t.getVideosByHashTag(hashTag=hashtag, count=count_per_seed)['body']['itemListData'])
  File "/home/claes/.local/share/virtualenvs/tiktok_parser-Bqk8ZjrR/lib/python3.8/site-packages/TikTokAPI/tiktokapi.py", line 157, in getVideosByHashTag
    return self.send_get_request(url, params, extra_headers=extra_headers)
  File "/home/claes/.local/share/virtualenvs/tiktok_parser-Bqk8ZjrR/lib/python3.8/site-packages/TikTokAPI/tiktokapi.py", line 41, in send_get_request
    data = get_req_json(url, params=None, headers=self.headers)
  File "/home/claes/.local/share/virtualenvs/tiktok_parser-Bqk8ZjrR/lib/python3.8/site-packages/TikTokAPI/utils.py", line 28, in get_req_json
    return json.loads(r.text)
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It is very strange error, cause sometimes it throwing, sometimes not. With the same parameters. Can you fix it?

upd: I tried to print(r.text) in TikTokAPI.utils.get_req_json and it shows normal json:

{"statusCode":0,"challengeInfo":{"challenge":{"id":"1638134384680965","title":"usatiktokers","desc":"","profileThumb":"","profileMedium":"","profileLarger":"","coverThumb":"","coverMedium":"","coverLarger":"","isCommerce":false},"stats":{"videoCount":1833,"viewCount":16000000},"shareMeta":{"title":"#usatiktokers on TikTok","desc":"16.0m views - Watch awesome short videos created with trending hashtag #usatiktokers"}}}

Also I've tried to make json.loads() this r.text and it still correct, but with your library not. Hmm..

upd2: Okay. I found that sometimes tiktok sends empty message:

def get_req_json(url, params=None, headers=None):
    r = requests.get(url, params=params, headers=headers)
    resp_str = r.content.decode()
    print(f'"{resp_str}"')
    resp = json.loads(r.content.decode())
    return resp

Output:

"{"statusCode":0,"challengeInfo":{"challenge":{"id":"1638134384680965","title":"usatiktokers","desc":"","profileThumb":"","profileMedium":"","profileLarger":"","coverThumb":"","coverMedium":"","coverLarger":"","isCommerce":false},"stats":{"videoCount":1832,"viewCount":16000000},"shareMeta":{"title":"#usatiktokers on TikTok","desc":"16.0m views - Watch awesome short videos created with trending hashtag #usatiktokers"}}}"
""

You can see that second message is empty.. And it is the same request. I'm requesting for usatiktokers hashtag only but it gets 2 response and one of them is empty. I see that problem in getVideosByHashTag return. First request is for getHashtag() and second (that empty) it is final request

rmarnold commented 4 years ago

TikTok is responding with an empty to 200 response. They must have changed the endpoint to another path.

avilash commented 4 years ago

There have been some changes in the parameters. Will fix it during the weekend. Cheers

avilash commented 4 years ago

Released a version with the fix Please run

pip install PyTikTokAPI --upgrade

Please verify so that we can close this

rmarnold commented 4 years ago

I need time to test, it looks like my IP has been blocked.

rmarnold commented 4 years ago

The issue seems to be fixed now. I had to append a proxy argument to test due to another non-related issue.

self.args.append("--proxy-server=")

Thanks,

avilash commented 4 years ago

Thanx

mikegrep commented 3 years ago

Hello I run into the same error TikTok api is the fanal version.How to fix that.Tanks