Closed JustAnotherArchivist closed 1 year ago
@PCJ-001 File a complete bug report if it isn't working. Make sure you're using the 'top' search order and the latest dev version.
I can get the code by running in google colab but not in my local pc. The error log is as follow
Error retrieving https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&q=%28from%3ADrZahidHamidi+OR+from%3AMuhyiddinYassin+OR+from%3AIsmailSabri60+OR+from%3Aanwaribrahim+OR+from%3Achedetofficial+OR+from%3AKhairykj+OR+from%3AHishammuddinH2O+OR+from%3AMukhrizMahathir+OR+from%3An_izzah+OR+from%3Atianchua%29+until%3A2023-01-14+since%3A2023-01-13&count=100&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&ext=mediaStats%2ChighlightedLabel: non-200 status code
4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&q=%28from%3ADrZahidHamidi+OR+from%3AMuhyiddinYassin+OR+from%3AIsmailSabri60+OR+from%3Aanwaribrahim+OR+from%3Achedetofficial+OR+from%3AKhairykj+OR+from%3AHishammuddinH2O+OR+from%3AMukhrizMahathir+OR+from%3An_izzah+OR+from%3Atianchua%29+until%3A2023-01-14+since%3A2023-01-13&count=100&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&ext=mediaStats%2ChighlightedLabel failed, giving up.
Traceback (most recent call last):
File "c:\Users\User\Documents\FYP Test\scrapingpoliticiantweet.py", line 83, in <module>
for tweet in sntwitter.TwitterSearchScraper(query,top=True).get_items():
File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\snscrape\modules\twitter.py", line 680, in get_items
for obj in self._iter_api_data('https://api.twitter.com/2/search/adaptive.json', params, paginationParams, cursor = self._cursor):
File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\snscrape\modules\twitter.py", line 369, in _iter_api_data
obj = self._get_api_data(endpoint, reqParams)
File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\snscrape\modules\twitter.py", line 339, in _get_api_data
r = self._get(endpoint, params = params, headers = self._apiHeaders, responseOkCallback = self._check_api_response)
File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\snscrape\base.py", line 216, in _get
return self._request('GET', *args, **kwargs)
File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\snscrape\base.py", line 212, in _request
raise ScraperException(msg)
snscrape.base.ScraperException: 4 requests to https://api.twitter.com/2/search/adaptive.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&cards_platform=Web-12&include_cards=1&include_ext_alt_text=true&include_quote_count=true&include_reply_count=1&tweet_mode=extended&include_entities=true&include_user_entities=true&include_ext_media_color=true&include_ext_media_availability=true&send_error_codes=true&simple_quoted_tweets=true&q=%28from%3ADrZahidHamidi+OR+from%3AMuhyiddinYassin+OR+from%3AIsmailSabri60+OR+from%3Aanwaribrahim+OR+from%3Achedetofficial+OR+from%3AKhairykj+OR+from%3AHishammuddinH2O+OR+from%3AMukhrizMahathir+OR+from%3An_izzah+OR+from%3Atianchua%29+until%3A2023-01-14+since%3A2023-01-13&count=100&query_source=spelling_expansion_revert_click&pc=1&spelling_corrections=1&ext=mediaStats%2ChighlightedLabel failed, giving up.
I have tried uninstalling and reinstalling the snscrape package. Hence is there any way to solve this?
Locking this to prevent further spam. A new release with this fix will probably follow soon, but if you're in a hurry, use the dev version (see README). If it still isn't working then, please file a complete separate bug report.
Since sometime today, 'top' searches fail entirely. The API endpoint returns 404.