search

shaikhsajid1111 / twitter-scraper-selenium

Python's package to scrap Twitter's front-end easily
https://pypi.org/project/twitter-scraper-selenium
MIT License
323 stars 50 forks source link

Tweets did not appear #76

Open corpuzdonn opened 1 year ago

corpuzdonn commented 1 year ago

Hello I keep getting this error. I am new to this package so i do not know what I am missing.

"Tweets did not appear!, Try setting headless=False to see what is happening"

ihabpalamino commented 1 year ago

having same problem

ansisme commented 1 year ago

same issue

mleita commented 1 year ago

Same!

ihabpalamino commented 1 year ago

@shaikhsajid1111 i have posted some questions in facebook scraping if u can help me there please

shaikhsajid1111 commented 1 year ago

What method are you using?, I assume profile scraping method should work but hashtag and keyword requires authentication

ansisme commented 1 year ago

I have used third party twitter login via Supabase, so I am inside the Twitter Logged in page for my profile, I need to scrape the data of the hashtags and keywords in order to achieve a functionality, but using this scraper I was not able to even tho i am logged in. Would you let me know how to scrape when logged in already ?

shaikhsajid1111 commented 1 year ago

@ansisme The feature was there initially but I removed it intentionally. This commit https://github.com/shaikhsajid1111/twitter-scraper-selenium/commit/9ce26412ccd38026de0abd88606618eb8d6c26c3 needs to be reverted for Python scripts, and than you can pass browser profile of logged in browser, it'll use that session, but do it at your own risks because that might lead to blocking of your account. One with self.profile and brower_profile variables should be focused for reverting keeping in mind that current selectors do not change