get_trends() not working [ KeyError: 'module_html']

Sue-Fwl commented 4 years ago

Greetings,

I'm getting an error on -> 'module_html' when calling 'get_trends()'


Traceback (most recent call last):

in
Trending = get_trends()

File "site-packages\twitter_scraper\modules\trends.py", line 17, in get_trends
html = html.json()["module_html"]

KeyError: 'module_html' ```

Svn-Sp commented 4 years ago

Does this error still occur? I can run get_trends() without any problem.

bisguzar commented 4 years ago

Please update @Sue-Fwl , also can you provide detailed informations? Like your region, VPN region if you are using vpn etc...

Sue-Fwl commented 4 years ago

Thanks for keeping the issue alive.

I just tried using all the modules: [I can still retrieve tweets but now both 'get_trends()' and 'Profile()' are not working, and the get_trends error code changed]

Traceback (most recent call last):

  File "test.py", line 41, in <module>
    get_trends()

  File "Anaconda3\lib\site-packages\twitter_scraper\modules\trends.py", line 44, in get_trends
    html = html.json()["module_html"]

  File "Anaconda3\lib\site-packages\requests\models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)

  File "\Anaconda3\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)

  File "\Anaconda3\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())

  File "\Anaconda3\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None

JSONDecodeError: Expecting value

I'm also getting this error for 'Profile()' -though it was working fine before- :

Traceback (most recent call last):

  File "<ipython-input-10-a1039dda4ede>", line 1, in <module>
    Profile('userprofile')

  File "\Anaconda3\lib\site-packages\twitter_scraper\modules\profile.py", line 36, in __init__
    self.__parse_profile(page)

  File "Anaconda3\lib\site-packages\twitter_scraper\modules\profile.py", line 49, in __parse_profile
    self.location = html.find(".ProfileHeaderCard-locationText")[0].text

UnboundLocalError: local variable 'html' referenced before assignment

I updated the modules to their latest GitHub versions and got the same error messages.

My location is in Western Asia and I'm not using a VPN. I don't think my region plays a role in this since 'get_tweets()' is running fine.

VatsalAgarwal commented 3 years ago

Issue still persists, I believe the code simply needs to be updated to parse the new JSON formats returned by the API. @bisguzar

bisguzar / twitter-scraper

get_trends() not working [ KeyError: 'module_html'] #158