shaikhsajid1111 / twitter-scraper-selenium

Python's package to scrap Twitter's front-end easily
https://pypi.org/project/twitter-scraper-selenium
MIT License
305 stars 47 forks source link

Get newest tweets with keyword #14

Open FsxShader2012 opened 2 years ago

FsxShader2012 commented 2 years ago

I'm using this awesome lib to get the newest tweets containing certain keywords. If you don't supply values for "since" and "until" you will get the same tweets everytime - the first tweets of the current day. I managed to get around that by deleting the since and until part from the search URL but that is certainly not intended 😄

Could you maybe add a live feature so your code doesn't get corrupted by me? 😆

shaikhsajid1111 commented 2 years ago

Yeah, I've set it to start searching keywords from the last 24h from the current time. I see. it's a good proposal, I'll implement it to find from the current time itself. You may expect this change from the next release 😄

FsxShader2012 commented 2 years ago

Thanks, appreciate it 😄 Another, question / suggestion: Time based searches? E.g. 2022-04-17-17:16:00

shaikhsajid1111 commented 2 years ago

It doesn't depends on me but Twitter, if they've time based searches, it'll be possible to implement 😅

FsxShader2012 commented 2 years ago

Seems possible: https://github.com/igorbrigadir/twitter-advanced-search/ Honestly, all the operators from the time class would be interesting to me 🤔

shaikhsajid1111 commented 2 years ago

Search keywords based on some time, yeah it is already available. You can pass since and until the argument to scrap tweets within some time range. For example:

from twitter_scraper_selenium import scrap_keyword
#scrap 10 posts by searching keyword "#nasa" from date 30th August,2021 till date 31st August,2021
nasa = scrap_keyword(keyword="#nasa", browser="firefox",
                      tweets_count=10,output_format="json" ,until="2021-08-31", since="2021-08-30")
print(nasa)
FsxShader2012 commented 2 years ago

True, I was speaking of these operators or am I just blind? 🙈

Class Operator Finds Tweets… Eg:
Time since_time:1142974200 On or after a specified unix timestamp in seconds. Combine with the "until" operator for dates between. Maybe easier to use than since_id below. 🔗
  until_time:1142974215 Before a specified unix timestamp in seconds. Combine with a "since" operator for dates between. Maybe easier to use than max_id below. 🔗
  since_id:tweet_id After (NOT inclusive) a specified Snowflake ID 🔗
  max_id:tweet_id At or before (inclusive) a specified Snowflake ID (see Note below) 🔗
  within_time:2d
within_time:3h
within_time:5m
within_time:30s
Search within the last number of days, hours, minutes, or seconds 🔗