I am trying to extract tweets for a given profile, but I got the error below:
here is my code:
def scrape_tweets(self, username):
for tweet in sntwitter.TwitterSearchScraper(f'from:{username}').get_items():
dal = DAL()
dal.insert_tweet(tweet=tweet)
self.scrape_replies(tweet=tweet)
It worked until at the mid of April 2023. Below is my error that I got:
Error retrieving https://api.twitter.com/1.1/guest/activate.json: non-200 response (401)
return self._request('POST', *args, **kwargs)
File "C:\Users\Gridi\Desktop\nlp\osfa2022\Source code\ASSA.Scraper.Twitter\snscrape\base.py", line 247, in _request
raise ScraperException(msg)
snscrape.base.ScraperException: 4 requests to https://api.twitter.com/1.1/guest/activate.json failed, giving up.
I know that twitter has applied restriction in its public APIs, but there is any fix for it ?
How to reproduce
def scrape_tweets(self, username):
for tweet in sntwitter.TwitterSearchScraper(f'from:{username}').get_items():
dal = DAL()
dal.insert_tweet(tweet=tweet)
self.scrape_replies(tweet=tweet)
Expected behaviour
To get the tweets
Screenshots and recordings
No response
Operating system
Windows 10
Python version: output of python3 --version
3.9
snscrape version: output of snscrape --version
latest version installed from pip
Scraper
TwitterSearchScraper
How are you using snscrape?
Module (import snscrape.modules.something in Python code)
Describe the bug
I am trying to extract tweets for a given profile, but I got the error below: here is my code:
It worked until at the mid of April 2023. Below is my error that I got:
I know that twitter has applied restriction in its public APIs, but there is any fix for it ?
How to reproduce
Expected behaviour
To get the tweets
Screenshots and recordings
No response
Operating system
Windows 10
Python version: output of
python3 --version
3.9
snscrape version: output of
snscrape --version
latest version installed from pip
Scraper
TwitterSearchScraper
How are you using snscrape?
Module (
import snscrape.modules.something
in Python code)Backtrace
No response
Log output
No response
Dump of locals
No response
Additional context
No response