medialab / minet

A webmining CLI tool & library for python.
GNU General Public License v3.0
281 stars 26 forks source link

tiktok search-videos error #972

Open csamson-sf opened 4 months ago

csamson-sf commented 4 months ago

I get the following error when using tiktok search-videos (for example : minet tiktok search-videos -l 400 "climate change scam" > test_climate_change_scam_tiktok.csv )

I used to get this error sometimes when doing a lot of consecutive searches but it worked again after a few hours but this time it's been a week and I still have this error on all the tiktok searches I try.

minet process was stopped because an error occurred!
Traceback (most recent call last):
  File "/home/charlotte/.local/bin/minet", line 8, in <module>
    sys.exit(main())
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/cli/__main__.py", line 14, in main
    run("minet", __identifier__, MINET_COMMANDS)
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/cli/utils.py", line 49, in wrapper
    fn(*args, **kwargs)
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/cli/run.py", line 127, in run
    fn(cli_args)
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/cli/utils.py", line 440, in wrapper
    return action(cli_args, *args, **additional_kwargs, **kwargs)
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/cli/tiktok/search_videos.py", line 31, in action
    for video in generator:
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/tiktok/api_scraper.py", line 95, in search_videos
    data = self.request_json(url)
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/web.py", line 1307, in decorated
    return retryer(fn, self, *args, **kwargs)
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/web.py", line 1201, in __call__
    return super().__call__(fn, *args, **kwargs)
  File "/home/charlotte/.local/lib/python3.10/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
  File "/home/charlotte/.local/lib/python3.10/site-packages/tenacity/__init__.py", line 314, in iter
    return fut.result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/charlotte/.local/lib/python3.10/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/tiktok/api_scraper.py", line 81, in request_json
    return response.json()
  File "/home/charlotte/.local/lib/python3.10/site-packages/minet/web.py", line 892, in json
    return json.loads(self.text())
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Yomguithereal commented 4 months ago

It seems tiktok has changed its public facing API. I need to investigate and see if we can patch things.

csamson-sf commented 3 months ago

Hi, just wondering if there were any updates ? Thank you !

Yomguithereal commented 3 months ago

Nope, sorry