taspinar / twitterscraper

Scrape Twitter for Tweets
MIT License
2.39k stars 581 forks source link

Sorry you are rate limited Message #333

Open ThomasADuffy opened 3 years ago

ThomasADuffy commented 3 years ago

Hello! So i am trying to run a query to pull all the tweets surrounding Mentioning Nike and i cant seem to do it.

I had it print out the json_resp on line 106 of the query.py just because it was error on the next line with a key error so that is the first bit. Im getting a message back saying sorry i am rate limited. any known work around? {'message': 'Sorry, you are rate limited.'} --- This is the json_resp.

Error Is below on each tweet

Failed to parse JSON while requesting "https://twitter.com/i/search/timeline?f=tweets&vertical=default&include_available_features=1&include_entities=1&reset_error_state=false&src=typd&max_position=TWEET-1290762499529244673-1290798381212807172&q=@UnderArmour%20since%3A2020-05-31%20until%3A2020-08-05&l=" Traceback (most recent call last): File "C:\Program Files\Anaconda3\lib\site-packages\twitterscraper\query.py", line 108, in query_single_page html = json_resp['items_html'] or '' KeyError: 'items_html'

This is my code below


json_lst=[]
for username in ['@Nike','@UnderArmour','@Adidas']:
       for tweet in query_tweets(username,begindate=dt.date(2017, 1, 1),enddate=dt.date.today(),poolsize=20, lang='', use_proxy=True):
            tweet.__dict__['timestamp'] = str(tweet.__dict__['timestamp'])
            json_lst.append(tweet)
        with open(f'{username}.json', 'a') as f:
            json.dump(json_lst,f, indent=4)
        json_lst = []```