Russell-Newton / TikTokPy

Extract data from TikTok without needing any login information or API keys.
https://pypi.org/project/tiktokapipy/
MIT License
197 stars 25 forks source link

Data is the same #20

Open smithperez opened 1 year ago

smithperez commented 1 year ago

I have been testing this API and notice that all the data returned is the same. I have been playing around with the challenge method at different periods of time and I notice that when you use a hashtag to gather some data and you use it a second time, it will return the same data as the first time regardless of the time lapse between the 2 requests.

Just in case, you can find the code attached below.


    def hashtags(self, hashtag: str='funny', video_count: int=10):
        """
        Retrun information about a TikTok hashtag
        including video

        hashtag: str
        video_count: int
        """
        now = datetime.now()
        current_time = now.strftime(f"%H_%M ({datetime.today().strftime('%Y-%m-%d')})")

        with TikTokAPI() as api:
            data = {}
            challenge = api.challenge(hashtag, video_limit=video_count, )

             # Evaluate if the user has a folder in the database_json folder
            if(not os.path.isdir(f"Backend/hashtag_database_json/{challenge.title}")):
                print(os.path.exists(f"Backend/hashtag_database_json/{challenge.title}"))
                os.makedirs(f'Backend/hashtag_database_json/{challenge.title}')
                print(f"Hashtag {challenge.title} has been created in the DB")

            for video in challenge.videos:

                #Store all the hashtags used in the video
                video_hashtags = []
                for i in video.challenges:
                    #Append all the challenges to the list
                    video_hashtags.append(i.title)

                data[video.id] = {
                    "likes":video.stats.digg_count,
                    "shares": video.stats.share_count,
                    "comments": video.stats.comment_count,
                    "play_count": video.stats.play_count,
                    "description": video.desc,

                    "video_info": {
                        "height": video.video.height,
                        "width": video.video.width,
                        "duration": video.video.duration,
                    },

                    "hashtags": video_hashtags,

                    "music_info": {
                        "title": video.music.title,
                        "author_name": video.music.author_name,
                        "original_audio": video.music.original,
                    }
                }

                print(video_hashtags)

                with open(f"Backend/hashtag_database_json/{challenge.title}/{current_time}.json", 'w') as f:
                    json.dump(data, f, indent=4)
Russell-Newton commented 1 year ago

I'll have to look into this more, but this problem arises because of how TikTok presents videos if you go to a tag's page.

TikTokPy simply visits the tag page to grab videos, but this means its subject to however TikTok wants to present the videos. TikTok decides video order by its algorithm, which means the videos aren't sorted by recent or even popular, just by what TikTok thinks you'd want to see. And because TikTokPy doesn't log in, this is a generic suggestion that doesn't change.

It's possible there's some API calls I could make to get recent videos tagged with a challenge, but at the moment I'm unsure if this is something I can do.

AlwynHD commented 1 year ago

Hi i have been using the code above to get a large array of tiktoks back however it throws an error:

    self._api.video(video.id) for video in converted.item_list
    For further information visit https://errors.pydantic.dev/2.0.1/v/missing
itemInfo.itemStruct.video.downloadAddr
  Field required [type=missing, input_value={'cover': 'https://p77-si...meInfo': {}, 'width': 0}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.0.1/v/missing
eelegiap commented 1 year ago

I'm having the same issue as @smithperez. Wish there was a way around this, but I'm not sure how. I guess it is not possible to sort videos on the tag page.

Russell-Newton commented 1 year ago

I'm having the same issue as @smithperez. Wish there was a way around this, but I'm not sure how. I guess it is not possible to sort videos on the tag page.

Yeah unless TikTok creates a way to sort on a tag page, this gets very complicated to do. I've got some ideas for how to grab random videos tagged with something, but I haven't had a chance to try it out yet.