search

taspinar / twitterscraper

Scrape Twitter for Tweets
MIT License
2.39k stars 581 forks source link

Twitterscraper was working fine till tuesday but now it shows 0 tweets #299

Closed lubhaniagarwal closed 4 years ago

lubhaniagarwal commented 4 years ago

I am facing problem while retrieving tweets with twitterscraper as it was working fine till tuesday but now it shows 0 tweets. I even tried to clone the git repository and change header list, but still it is not working. Is there anyway to fix this issue or anything wrong on twitterscraper side. please help me out !!:((

hakanyusufoglu commented 4 years ago

Yes me too. Problem is back. Please help !

lapp0 commented 4 years ago

Please share the command you're running, and any input files so I can try to reproduce.

lubhaniagarwal commented 4 years ago

first installed twitterscraper using pip install twitterscraper. It was working fine till tuesday. then i uninstalled it cloned repo-https://github.com/taspinar/twitterscraper.git changed the header list in query.py to :HEADERS_LIST = ['Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36'] then run - python setup.py install

but its not working.

command that i am working on is:

twitterscraper "amazon delivery" -bd 2010-01-01 -ed 2020-05-26 --limit 100000 --lang en --output xyz.json

thanks a lot.

lapp0 commented 4 years ago

I ran it from origin/master without making any changes to the headers and got 653 results. Could you share the programs full output?

lubhaniagarwal commented 4 years ago

error.txt here is the commands I'm running on command prompt and results that i get . thank you

lapp0 commented 4 years ago

I'm not sure why I was able to retrieve tweets. It appears other users are suggesting twitter doesn't allow JS disabled.

https://github.com/taspinar/twitterscraper/issues/296#issuecomment-638791009

I will try to hack together a selenium solution tonight.

lubhaniagarwal commented 4 years ago

Okayy. Thank you so much for your inputs.

On Fri, Jun 5, 2020, 01:56 lapp0 notifications@github.com wrote:

I'm not sure why I was able to retrieve tweets. It appears other users are suggesting twitter doesn't allow JS disabled.

296 (comment)

https://github.com/taspinar/twitterscraper/issues/296#issuecomment-638791009

I will try to hack together a selenium solution tonight.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/taspinar/twitterscraper/issues/299#issuecomment-639097767, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKUJE5HZMQJIZAEC2COPAXLRU77OZANCNFSM4NSO6X2Q .

lapp0 commented 4 years ago

Please try this and let me know how it works for you https://github.com/taspinar/twitterscraper/pull/302

abhisheksaxena1998 commented 4 years ago

HEADER = {'User-Agent': random.choice(HEADERS_LIST), 'X-Requested-With': 'XMLHttpRequest'}

This line in query.py will surely fix this issue.

lubhaniagarwal commented 4 years ago

@abhisheksaxena1998 thanks a lot .It started working now. @lapp0 I will try this #302 and let you know. Thanks you so much! :)

lapp0 commented 4 years ago

Duplicate of https://github.com/taspinar/twitterscraper/issues/296

lapp0 commented 4 years ago

@lubhaniagarwal other users are suggesting that legacy now works, give origin/master another try