shaikhsajid1111 / twitter-scraper-selenium

Python's package to scrap Twitter's front-end easily
https://pypi.org/project/twitter-scraper-selenium
MIT License
323 stars 50 forks source link

Returns only "Tweets did not appear!" #20

Closed Weltolk closed 2 years ago

Weltolk commented 2 years ago

Environment Information

OS Version:

Edition Windows 10 Pro
Version 21H2
Installed on    ‎2022.‎04.‎10
OS build    19044.1706
Experience  Windows Feature Experience Pack 120.2212.4170.0

Python Version:

PS C:\Users\xxx> python -V
Python 3.10.5

Twitter scraper selenium Version:

0.1.6

requirement.txt is already installed

Code

from twitter_scraper_selenium import scrap_profile

microsoft = scrap_profile(twitter_username="Microsoft", output_format="json", browser="firefox", tweets_count=10)
print(microsoft)

Error Information

C:\Users\xxx\PycharmProjects\venv\Scripts\python.exe
C:/Users/xxx/PycharmProjects/Twitter_Selenium.py
[WDM] - Driver [C:\Users\xxx\.wdm\drivers\geckodriver\win64\v0.31.0\geckodriver.exe] found in cache
Tweets did not appear!
Tweets did not appear!
Tweets did not appear!
Tweets did not appear!
Tweets did not appear!
Tweets did not appear!
Tweets did not appear!
Tweets did not appear!
Tweets did not appear!
Tweets did not appear!
{}

Process finished with exit code 0

log

geckodriver.log:

1655093631147   geckodriver INFO    Listening on 127.0.0.1:23023
1655093631171   mozrunner::runner   INFO    Running command: "C:\\Program Files\\Mozilla Firefox\\firefox.exe" "--marionette" "--headless" "--no-sandbox" "--disable-dev-shm-usage" "--ignore-certificate-errors" "--disable-gpu" "--log-level=3" "--disable-notifications" "--disable-popup-blocking" "-no-remote" "-profile" "C:\\Users\\xxx\\AppData\\Local\\Temp\\rust_mozprofileMChkmM"
*** You are running in headless mode.
1655093631395   Marionette  INFO    Marionette enabled
[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt
console.warn: SearchSettings: "get: No settings file exists, new profile?" (new NotFoundError("Could not open the file at C:\\Users\\xxx\\AppData\\Local\\Temp\\rust_mozprofileMChkmM\\search.json.mozlz4", (void 0)))
1655093632149   Marionette  INFO    Listening on port 23060
Read port: 23060
1655093632179   RemoteAgent WARN    TLS certificate errors will be ignored for this session
1655093632180   RemoteAgent INFO    Proxy settings initialised: {"proxyType":"manual","httpProxy":"127.0.0.1:23022","noProxy":[],"sslProxy":"127.0.0.1:23022"}
1655093781898   Marionette  INFO    Stopped listening on port 23060

###!!! [Parent][PImageBridgeParent] Error: RunMessage(msgname=PImageBridge::Msg_WillClose) Channel closing: too late to send/recv, messages will be lost
shaikhsajid1111 commented 2 years ago

Thank you so much @Weltolk for raising this issue and those 2 pull requests 🙂

Weltolk commented 2 years ago

Thank you so much @Weltolk for raising this issue and those 2 pull requests 🙂

pleasure