Russell-Newton / TikTokPy

Extract data from TikTok without needing any login information or API keys.
https://pypi.org/project/tiktokapipy/
MIT License
192 stars 24 forks source link

[QUESTION] Any way to keep track and resume from cursor? #66

Closed TonyWilly closed 11 months ago

TonyWilly commented 11 months ago

Ask your question I'd like to fetch all the videos from hashtags that have quite a few videos and possibly from sounds as well. I'm wondering if I can keep track of the current cursor or challenge object or api so that I can stop and resume from the same spot. Or maybe if I could supply a starting cursor to concurrently get the data from the same challenge faster? Thanks.

Version Information Please include what versions of pydantic, playwright, and tiktokapipy you have installed (can be found with pip freeze). pydantic==2.1.1 pydantic_core==2.4.0 playwright==1.36.0 tiktokapipy==0.2.2

System Information Windows 10 64-bit Desktop

Region Information Canada, using US proxies at the moment

Additional context

Russell-Newton commented 11 months ago

The challenge.videos, user.videos, and video.comments iterators all keep track of the current cursor internally. While it's a bit hacky, what you could do is something like this:

challenge.videos._cursor = 30  # or whatever you want
for video in challenge.videos:
    ...

I haven't tried this, so you might get some errors if you set the cursor to something that doesn't work. Notably, user.videos expects a unix timestamp in milliseconds (to grab a certain number of videos posted before that time), all other cursors are just based on how many items you want to skip over.

TonyWilly commented 11 months ago

I think this will work for me, thanks!