search

shaikhsajid1111 / twitter-scraper-selenium

Python's package to scrap Twitter's front-end easily
https://pypi.org/project/twitter-scraper-selenium
MIT License
308 stars 50 forks source link

scrap hashtag #38

Open toastyy00 opened 2 years ago

toastyy00 commented 2 years ago

does using hashtags work? I want data on people who use a specific hashtag and find out who uses it the most

shaikhsajid1111 commented 2 years ago

Yeah, hashtag works. You can use scrap_keyword method to achieve that. Code should look something like

from twitter_scraper_selenium import scrap_keyword

scrap_keyword(keyword="#YOUR_HASHTAG", browser="firefox",tweets_count=10,output="csv",filename="YOUR_FILENAME")
toastyy00 commented 2 years ago

cool. is it possible to scrape the whole hashtag in one day?

shaikhsajid1111 commented 2 years ago

It depends on number of tweets available for that hashtag

toastyy00 commented 2 years ago

okay so is it okay to input 9999 on "tweets_count" to scrap it all?

shaikhsajid1111 commented 2 years ago

I think you might get blocked at some point of time while scraping that amount. Couple of better approach you can use,

  1. In case you get blocked scraping data in unauthenticated way, the crawler will exit itself and give you the data whatever was scraped. You can pass the id of the last tweet under the argument since_id. See what all feature you can use here .
  2. You can login using some Twitter account and pass browser profile and scrap in authenticated way. Beware that your account may get blocked as well.