bisguzar / twitter-scraper

Scrape the Twitter Frontend API without authentication.
MIT License
3.88k stars 601 forks source link

get_tweets in your example is not working #192

Closed etam89 closed 2 years ago

etam89 commented 2 years ago

from twitter_scraper import get_tweets for tweet in get_tweets('twitter',pages=1): print(tweet['text'])

error message:

JSONDecodeError Traceback (most recent call last)

in 1 from twitter_scraper import get_tweets ----> 2 for tweet in get_tweets('twitter',pages=1): 3 print(tweet['text']) /usr/local/lib/python3.6/dist-packages/twitter_scraper/modules/tweets.py in get_tweets(query, pages) 168 pages += -1 169 --> 170 yield from gen_tweets(pages) 171 172 /usr/local/lib/python3.6/dist-packages/twitter_scraper/modules/tweets.py in gen_tweets(pages) 35 try: 36 html = HTML( ---> 37 html=r.json()["items_html"], url="bunk", default_encoding="utf-8" 38 ) 39 except KeyError: /usr/local/lib/python3.6/dist-packages/requests/models.py in json(self, **kwargs) 898 # used. 899 pass --> 900 return complexjson.loads(self.text, **kwargs) 901 902 @property /usr/local/lib/python3.6/dist-packages/simplejson/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, **kw) 523 parse_constant is None and object_pairs_hook is None 524 and not use_decimal and not kw): --> 525 return _default_decoder.decode(s) 526 if cls is None: 527 cls = JSONDecoder /usr/local/lib/python3.6/dist-packages/simplejson/decoder.py in decode(self, s, _w, _PY3) 368 if _PY3 and isinstance(s, bytes): 369 s = str(s, self.encoding) --> 370 obj, end = self.raw_decode(s) 371 end = _w(s, end).end() 372 if end != len(s): /usr/local/lib/python3.6/dist-packages/simplejson/decoder.py in raw_decode(self, s, idx, _w, _PY3) 398 elif ord0 == 0xef and s[idx:idx + 3] == '\xef\xbb\xbf': 399 idx += 3 --> 400 return self.scan_once(s, idx=_w(s, idx).end()) JSONDecodeError: Expecting value: line 1 column 1 (char 0)
bisguzar commented 2 years ago

duplication of #189