Closed couldbejake closed 1 year ago
The tweet scraper can not search by hashtag
snscrape --max-results 100 twitter-hashtag archiveteam
100 tweets about the #archiveteam hashtag
No response
Windows 11
python3 --version
Python 3.8
snscrape --version
snscrape 0.6.2.20230320
Twitter-scraper
CLI (snscrape ... as a command, e.g. in a terminal)
snscrape ...
PS C:\Users\jakenelson\Desktop\api twitter 3> snscrape --max-results 100 twitter-hashtag archiveteam 2023-06-19 16:15:43.882 ERROR snscrape.base Error retrieving https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&include_ext_has_nft_avatar=1&include_ext_is_blue_verified=1&include_ext_verified_type=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_ext_limited_action_results=false&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_ext_collab_control=true&include_ext_views=true&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&include_ext_sensitive_media_warning=true&include_ext_trusted_friends_metadata=true&send_error_codes=true&simple_quoted_tweet=true&q=%23archiveteam&tweet_search_mode=live&count=20&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&include_ext_edit_control=true&ext=mediaStats%2ChighlightedLabel%2ChasNftAvatar%2CvoiceInfo%2Cenrichments%2CsuperFollowMetadata%2CunmentionInfo%2CeditControl%2Ccollab_control%2Cvibe: blocked (403) 2023-06-19 16:15:43.883 CRITICAL snscrape.base 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&include_ext_has_nft_avatar=1&include_ext_is_blue_verified=1&include_ext_verified_type=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_ext_limited_action_results=false&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_ext_collab_control=true&include_ext_views=true&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&include_ext_sensitive_media_warning=true&include_ext_trusted_friends_metadata=true&send_error_codes=true&simple_quoted_tweet=true&q=%23archiveteam&tweet_search_mode=live&count=20&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&include_ext_edit_control=true&ext=mediaStats%2ChighlightedLabel%2ChasNftAvatar%2CvoiceInfo%2Cenrichments%2CsuperFollowMetadata%2CunmentionInfo%2CeditControl%2Ccollab_control%2Cvibe failed, giving up. 2023-06-19 16:15:43.884 CRITICAL snscrape.base Errors: blocked (403), blocked (403), blocked (403), blocked (403) 2023-06-19 16:15:43.929 CRITICAL snscrape._cli Dumped stack and locals to C:\Users\JAKENE~1\AppData\Local\Temp\snscrape_locals_0rk5epvx Traceback (most recent call last): File "C:\Users\jakenelson\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\jakenelson\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\jakenelson\AppData\Local\Programs\Python\Python38\Scripts\snscrape.exe__main__.py", line 7, in File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape_cli.py", line 320, in main for i, item in enumerate(scraper.get_items(), start = 1): File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\modules\twitter.py", line 1661, in get_items for obj in self._iter_api_data('https://api.twitter.com/2/search/adaptive.json', _TwitterAPIType.V2, params, paginationParams, cursor = self._cursor): File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\modules\twitter.py", line 761, in _iter_api_data obj = self._get_api_data(endpoint, apiType, reqParams) File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\modules\twitter.py", line 727, in _get_api_data r = self._get(endpoint, params = params, headers = self._apiHeaders, responseOkCallback = self._check_api_response) File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\base.py", line 251, in _get return self._request('GET', *args, **kwargs) File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\base.py", line 247, in _request raise ScraperException(msg) snscrape.base.ScraperException: 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&include_ext_has_nft_avatar=1&include_ext_is_blue_verified=1&include_ext_verified_type=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_ext_limited_action_results=false&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_ext_collab_control=true&include_ext_views=true&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&include_ext_sensitive_media_warning=true&include_ext_trusted_friends_metadata=true&send_error_codes=true&simple_quoted_tweet=true&q=%23archiveteam&tweet_search_mode=live&count=20&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&include_ext_edit_control=true&ext=mediaStats%2ChighlightedLabel%2ChasNftAvatar%2CvoiceInfo%2Cenrichments%2CsuperFollowMetadata%2CunmentionInfo%2CeditControl%2Ccollab_control%2Cvibe failed, giving up.
JustAnotherArchivist
commented
1 year ago JustAnotherArchivist
commented
1 year ago
Describe the bug
The tweet scraper can not search by hashtag
How to reproduce
snscrape --max-results 100 twitter-hashtag archiveteam
Expected behaviour
100 tweets about the #archiveteam hashtag
Screenshots and recordings
No response
Operating system
Windows 11
Python version: output of
python3 --versionPython 3.8
snscrape version: output of
snscrape --versionsnscrape 0.6.2.20230320
Scraper
Twitter-scraper
How are you using snscrape?
CLI (
snscrape ...as a command, e.g. in a terminal)Backtrace
PS C:\Users\jakenelson\Desktop\api twitter 3> snscrape --max-results 100 twitter-hashtag archiveteam 2023-06-19 16:15:43.882 ERROR snscrape.base Error retrieving https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&include_ext_has_nft_avatar=1&include_ext_is_blue_verified=1&include_ext_verified_type=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_ext_limited_action_results=false&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_ext_collab_control=true&include_ext_views=true&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&include_ext_sensitive_media_warning=true&include_ext_trusted_friends_metadata=true&send_error_codes=true&simple_quoted_tweet=true&q=%23archiveteam&tweet_search_mode=live&count=20&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&include_ext_edit_control=true&ext=mediaStats%2ChighlightedLabel%2ChasNftAvatar%2CvoiceInfo%2Cenrichments%2CsuperFollowMetadata%2CunmentionInfo%2CeditControl%2Ccollab_control%2Cvibe: blocked (403) 2023-06-19 16:15:43.883 CRITICAL snscrape.base 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&include_ext_has_nft_avatar=1&include_ext_is_blue_verified=1&include_ext_verified_type=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_ext_limited_action_results=false&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_ext_collab_control=true&include_ext_views=true&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&include_ext_sensitive_media_warning=true&include_ext_trusted_friends_metadata=true&send_error_codes=true&simple_quoted_tweet=true&q=%23archiveteam&tweet_search_mode=live&count=20&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&include_ext_edit_control=true&ext=mediaStats%2ChighlightedLabel%2ChasNftAvatar%2CvoiceInfo%2Cenrichments%2CsuperFollowMetadata%2CunmentionInfo%2CeditControl%2Ccollab_control%2Cvibe failed, giving up. 2023-06-19 16:15:43.884 CRITICAL snscrape.base Errors: blocked (403), blocked (403), blocked (403), blocked (403) 2023-06-19 16:15:43.929 CRITICAL snscrape._cli Dumped stack and locals to C:\Users\JAKENE~1\AppData\Local\Temp\snscrape_locals_0rk5epvx Traceback (most recent call last): File "C:\Users\jakenelson\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\jakenelson\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\jakenelson\AppData\Local\Programs\Python\Python38\Scripts\snscrape.exe__main__.py", line 7, in
File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape_cli.py", line 320, in main
for i, item in enumerate(scraper.get_items(), start = 1):
File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\modules\twitter.py", line 1661, in get_items
for obj in self._iter_api_data('https://api.twitter.com/2/search/adaptive.json', _TwitterAPIType.V2, params, paginationParams, cursor = self._cursor):
File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\modules\twitter.py", line 761, in _iter_api_data
obj = self._get_api_data(endpoint, apiType, reqParams)
File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\modules\twitter.py", line 727, in _get_api_data
r = self._get(endpoint, params = params, headers = self._apiHeaders, responseOkCallback = self._check_api_response)
File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\base.py", line 251, in _get
return self._request('GET', *args, **kwargs)
File "c:\users\jakenelson\appdata\local\programs\python\python38\lib\site-packages\snscrape\base.py", line 247, in _request
raise ScraperException(msg)
snscrape.base.ScraperException: 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&include_ext_has_nft_avatar=1&include_ext_is_blue_verified=1&include_ext_verified_type=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_ext_limited_action_results=false&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_ext_collab_control=true&include_ext_views=true&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&include_ext_sensitive_media_warning=true&include_ext_trusted_friends_metadata=true&send_error_codes=true&simple_quoted_tweet=true&q=%23archiveteam&tweet_search_mode=live&count=20&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&include_ext_edit_control=true&ext=mediaStats%2ChighlightedLabel%2ChasNftAvatar%2CvoiceInfo%2Cenrichments%2CsuperFollowMetadata%2CunmentionInfo%2CeditControl%2Ccollab_control%2Cvibe failed, giving up.
Log output
No response
Dump of locals
No response
Additional context
No response