JustAnotherArchivist / snscrape

A social networking service scraper in Python
GNU General Public License v3.0
4.52k stars 712 forks source link

Stopping after 20 empty pages warning with the new dev version of snscrape.modules.twitter #922

Closed nirmals-workspace closed 1 year ago

nirmals-workspace commented 1 year ago

Describe the bug

In the below code scrape_tweets is a function which just fetches whatever things I need and return a list of scraped content from one tweet.... Now when I execute the following, I am getting a warning which goes like <Stopping after 20 empty pages> and the tweet_list remains an empty list

How to reproduce

tweets_list = []
for i,tweet in enumerate(sntwitter.TwitterSearchScraper('data since:2023/02/18 until:2023/05/18').get_items()):  
                if i > 9:
                    break
                scrapped_tweet = scrape_tweets(tweet)
                tweets_list.append(scrapped_tweet)

Expected behaviour

It won't throw any error but giving out a warning saying Stopping after 20 empty pages

Screenshots and recordings

No response

Operating system

Windows11

Python version: output of python3 --version

Python 3.10

snscrape version: output of snscrape --version

The new dev version

Scraper

TwitterSearchScraper

How are you using snscrape?

Module (import snscrape.modules.something in Python code)

Backtrace

No errors but warning and returnning nothing

Log output

No response

Dump of locals

No response

Additional context

No response

yemregundogmus commented 1 year ago

Hello, Change your query like this; data since:2023-02-18 until:2023-05-18 Twitter cant handle "/"

nirmals-workspace commented 1 year ago

Thanks for the clarification @yemregundogmus