Closed ghost closed 1 year ago
That error is impossible with version 0.7.0. You are using something older.
i have encountered that problem too, i am using snscrape 0.7.0.20230622
@doffn No, as I said, that exact error is impossible with version 0.7.0. You probably have two (or more) installations of snscrape and are using an older one than you think.
Describe the bug
Hello, I'm trying to scrape tweets for the last 3 months mentioning a certain keyword. It used to work fine now it doesn't. I'm using Python.
How to reproduce
import snscrape.modules.twitter as sntwitter
for index, tweet in enumerate(sntwitter.TwitterSearchScraper(str(fetching_keyword).strip() + ' since:' + starting_date + ' until:' + end_date, maxEmptyPages = 99999999).get_items()): tweets_list.append([tweet.id, tweet.date, tweet.user.username, tweet.user.location, tweet.lang, tweet.content, tweet.replyCount, tweet.retweetCount, tweet.likeCount, tweet.quoteCount, tweet.outlinks, tweet.tcooutlinks, tweet.retweetedTweet, tweet.quotedTweet, tweet.mentionedUsers])
Expected behaviour
Tweets should be scraped as normal but instead, I receive an error.
Screenshots and recordings
No response
Operating system
Windows 11
Python version: output of
python3 --version
3.8.16
snscrape version: output of
snscrape --version
0.7.0.20230622
Scraper
TwitterSearchScraper
How are you using snscrape?
Module (
import snscrape.modules.something
in Python code)Backtrace
No response
Log output
No response
Dump of locals
ScraperException: 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&include_ext_has_nft_avatar=1&include_ext_is_blue_verified=1&include_ext_verified_type=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_ext_limited_action_results=false&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_ext_collab_control=true&include_ext_views=true&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&include_ext_sensitive_media_warning=true&include_ext_trusted_friends_metadata=true&send_error_codes=true&simple_quoted_tweet=true&q=Bank+of+Canada+since%3A2023-03-20+until%3A2023-6-25&tweet_search_mode=live&count=20&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&include_ext_edit_control=true&ext=mediaStats%2ChighlightedLabel%2ChasNftAvatar%2CvoiceInfo%2Cenrichments%2CsuperFollowMetadata%2CunmentionInfo%2CeditControl%2Ccollab_control%2Cvibe failed, giving up.
Additional context
No response