iSarabjitDhiman / TweeterPy

TweeterPy is a python library to extract data from Twitter. TweeterPy API lets you scrape data from a user's profile like username, userid, bio, followers/followings list, profile media, tweets, etc.
MIT License
145 stars 20 forks source link

Couldn't generate guest token #40

Closed fpmirabile closed 9 months ago

fpmirabile commented 11 months ago

Hey, how are you?

I've been off for a while, working on my final engineering project. I'm having this issue on my EC2 instance:

2023-10-28 19:35:52,979 [ERROR] :: Couldn't generate a new session.
'guest_token'
Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.9/site-packages/tweeterpy/tweeterpy.py", line 152, in generate_session
    guest_token = make_request(
KeyError: 'guest_token'
Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/ec2-user/flask_api/phishbuster-cron/src/api/api.py", line 19, in <module>
    twitter_collector = TwitterDataCollector()
  File "/home/ec2-user/flask_api/phishbuster-cron/src/lib_tweeterpy/scrap_tweeterpy.py", line 12, in __init__
    self.twitter = TweeterPy()
  File "/home/ec2-user/.local/lib/python3.9/site-packages/tweeterpy/tweeterpy.py", line 29, in __init__
    self.generate_session()
  File "/home/ec2-user/.local/lib/python3.9/site-packages/tweeterpy/tweeterpy.py", line 152, in generate_session
    guest_token = make_request(
KeyError: 'guest_token'

I really dunno why it is happening and running out of options tbh. Could you help me @iSarabjitDhiman ?

iSarabjitDhiman commented 11 months ago

Hey @fpmirabile I am good thanks. Hope you are well.

Have you tried residential proxies? The script usually doesn't work on VPS without proxies. Also check if the script is running fine on your local machine. If it is, try using it with residential proxies on your EC2 instance.

If it still doesn't work, let me know.

fpmirabile commented 11 months ago

I'm well thanks ! Yes you are right, after reading and trying to udnerstand I saw a 429 (Too many requests) there. Do you have any suggestion for the proxy? I never used one, and I'd love to avoid this issue, since probably today or tomorrow I will be doing a demo and need the "scrapper" to work

iSarabjitDhiman commented 11 months ago

I'm well thanks ! Yes you are right, after reading and trying to udnerstand I saw a 429 (Too many requests) there. Do you have any suggestion for the proxy? I never used one, and I'd love to avoid this issue, since probably today or tomorrow I will be doing a demo and need the "scrapper" to work

You can use any of them. Just search for residential proxies. I have used brightdata and smartproxy before. These work pretty well. (I am not affiliated with any of these.) There are also services like oxylabs, soax etc.

Do your research before choosing any. Feel free to close the issue as well.