search

davidteather / TikTok-Api

The Unofficial TikTok API Wrapper In Python
https://davidteather.github.io/TikTok-Api
MIT License
4.82k stars 967 forks source link

[FEATURE_REQUEST] - Filtering by time #850

Closed benostarzec closed 1 year ago

benostarzec commented 2 years ago

I'd really like if there was a way to filter videos by the date they were posted on (e.g. query for videos about a keyword that have been posted today, or since january 1, etc). It would be great if this were an option built into the API, but I'd also gladly take any workarounds anyone's come up with for this in general. I also thought maybe TikTok'd have some kind of special search syntax/operators that you could send with the string keyword request, but did some googling and some experimenting and didn't find any so i'd be happy if somebody knew something about that too. I know that I could just filter the videos in python using the time information but that would still require requesting and processing all the videos first which isn't feasible, so ideally I'd have a solution that has tiktok do that so the invalid videos never even make it to me.

MohamedZribi commented 2 years ago

In the mobile app....there is a feature to filter videos of certain keywords and hashtags by standard timeframes such as last 24 hours or last week and so on....I tried to reverse engineer the request in my android phone using mitmproxy...I did the same thing for reddit not long ago for a diff task...so the issue for me was that since Android disabled certificate pinning...I had to patch every single app separately...while the operation was successful for reddit, it was not with Tiktok...so if you have an IOS machine....you may do this... But I'm also interested in this issue...if anybody have further ideas.

benostarzec commented 2 years ago

ok, i do have an IOS machine, i'm not exactly sure how i'd go about reverse engineering the requests like that but i will look into it if i can't come up with a more straightforeward solution, thanks.

benostarzec commented 2 years ago

@wait3 I kinda got it to work with normal requests on my laptop but i'm not sure how i could read requests sent on my iphone so that we can figure out the mobile app's filters, do you know?

MohamedZribi commented 2 years ago

@benostarzec You see...requests that you see on the browser dev tools are no problem to read and extract, but the web interface of Tiktok doesn't offer that feature of sorting videos by a specific timeframe....The mobile app however allow this to a certain extent....So the challenge is that requests sent from and to the mobile app, go through a private mobile API of Tiktok....in that case you'll have to reverse engineer this API to extract the requests... you'll need a proxy in the middle...I've used mitmproxy....mitmweb exactly where requests will show up in your laptop browser.

Check this video for reference: https://www.youtube.com/watch?v=RchCi6E2hVs

Also...I'd appreciate it if you could keep me in the loop, in case this turns out to be fruitful.

benostarzec commented 2 years ago

@wait3 Alright, thanks for your response. I'll go about trying to get this to work, and I'll also keep you updated :) Good luck

Lackaday9 commented 2 years ago

In the mobile app....there is a feature to filter videos of certain keywords and hashtags by standard timeframes such as last 24 hours or last week and so on....I tried to reverse engineer the request in my android phone using mitmproxy...I did the same thing for reddit not long ago for a diff task...so the issue for me was that since Android disabled certificate pinning...I had to patch every single app separately...while the operation was successful for reddit, it was not with Tiktok...so if you have an IOS machine....you may do this... But I'm also interested in this issue...if anybody have further ideas.

As far as I'm aware IOS is harder to root than Android. So what does IOS do that makes reverse engineering easier?

MohamedZribi commented 2 years ago

In the mobile app....there is a feature to filter videos of certain keywords and hashtags by standard timeframes such as last 24 hours or last week and so on....I tried to reverse engineer the request in my android phone using mitmproxy...I did the same thing for reddit not long ago for a diff task...so the issue for me was that since Android disabled certificate pinning...I had to patch every single app separately...while the operation was successful for reddit, it was not with Tiktok...so if you have an IOS machine....you may do this... But I'm also interested in this issue...if anybody have further ideas.

As far as I'm aware IOS is harder to root than Android. So what does IOS do that makes reverse engineering easier?

I guess, in newer versions, Android disabled certificate pinning and now you need to patch every app separately to monitor requests, while it's not the case with IOS, to the best of my knowledge.

Influencerdata commented 2 years ago

Hey we have the API for scarping data and offer free token to try out, Our API does fetch real-time data from social media like TikTok, twitter, twitch, reddit and youtube! you can check it out to see if this works for you https://www.influencerhunters.com/docs.html

davidteather commented 1 year ago

You can filter after