Russell-Newton / TikTokPy

Extract data from TikTok without needing any login information or API keys.
https://pypi.org/project/tiktokapipy/
MIT License
192 stars 24 forks source link

[BUG] Iterate over user videos not working #81

Open stolenvw opened 7 months ago

stolenvw commented 7 months ago

can no longer iterate over users videos,

doing user = await api.user(user_tag) print(user) returns

unique_id='user_tag' id=00000000000 nickname='user_tag' sec_uid='MS4wLjABAAAAfLpKG68lfXPA_bObhf2vKGJcFgdjffU5K7ObS7xz_Wg6oZZIZkP0lbnq-nWGtnZv' private_account=False verified=False stats=UserStats(follower_count=1056, following_count=1352, heart_count=4932, video_count=78, digg_count=0)

"id and names removed"

doing a print(user.videos) returns <tiktokapipy.util.deferred_collectors.DeferredItemListIterator object at 0x7f5044f4a440>

doing async for video in user.videos: print(video.id) just hangs

Im in the US

Python 3.10.7
tiktokapipy==0.2.4.post2
playwright==1.32.1

looking at my log last time it worked was on November 21

Jedeiah commented 7 months ago

I also encountered the same problem. It was working a few months ago, but now it doesn’t.

Russell-Newton commented 7 months ago

I think the user function might just be nuked everywhere at the moment. TikTok changed how they handle user pages within the EU, and it's possible they've rolled it out to the rest of the world too. I'll need to look into this more. Related to #77 and #80

stolenvw commented 7 months ago

I think the user function might just be nuked everywhere at the moment. TikTok changed how they handle user pages within the EU, and it's possible they've rolled it out to the rest of the world too. I'll need to look into this more. Related to #77 and #80

yeah right now im using the data_dump_file option and just pulling the info i want from the json dump

GDRAK23 commented 6 months ago

The problem still persists, I'm running this now and got this error:

/home/usr/.local/lib/python3.8/site-packages/tiktokapipy/api.py:143:

TikTokAPIWarning: Reached navigation timeout. Retrying...response = self._scrape_data( Traceback (most recent call last): File "", line 1, in File "", line 3, in get_video_info File "/home/usr/.local/lib/python3.8/site-packages/tiktokapipy/api.py", line 143, in user response = self._scrape_data( File "/home/usr/.local/lib/python3.8/site-packages/tiktokapipy/api.py", line 270, in _scrape_data raise TikTokAPIError(

tiktokapipy.TikTokAPIError: Data scraping unable to complete in 30.0s (retries: 0)

May be they found out and are now blocking the scrapping...

GDRAK23 commented 6 months ago

I think the user function might just be nuked everywhere at the moment. TikTok changed how they handle user pages within the EU, and it's possible they've rolled it out to the rest of the world too. I'll need to look into this more. Related to #77 and #80

yeah right now im using the data_dump_file option and just pulling the info i want from the json dump

I think the user function might just be nuked everywhere at the moment. TikTok changed how they handle user pages within the EU, and it's possible they've rolled it out to the rest of the world too. I'll need to look into this more. Related to #77 and #80

yeah right now im using the data_dump_file option and just pulling the info i want from the json dump

how do you use this 'dump_data_file' could you help me please?

stolenvw commented 6 months ago

I think the user function might just be nuked everywhere at the moment. TikTok changed how they handle user pages within the EU, and it's possible they've rolled it out to the rest of the world too. I'll need to look into this more. Related to #77 and #80

yeah right now im using the data_dump_file option and just pulling the info i want from the json dump

I think the user function might just be nuked everywhere at the moment. TikTok changed how they handle user pages within the EU, and it's possible they've rolled it out to the rest of the world too. I'll need to look into this more. Related to #77 and #80

yeah right now im using the data_dump_file option and just pulling the info i want from the json dump

how do you use this 'dump_data_file' could you help me please?

that was a work around for getting 'user.videos' info, there is no fix i know of for data scraping being broken

Russell-Newton commented 6 months ago

This is now related to #82. TikTok changed their data model, and I have yet to update this library to match.