search

1220moritz / reverse-twitter-scraper

A request based package for scraping twitter data. No API Key required. Support for proxies and private twitter accounts
Discord: moritz#1220
MIT License
26 stars 3 forks source link

Loading Profiles and cookies instead of statically setting cookie #1

Closed krjii closed 1 year ago

krjii commented 1 year ago

Instead of the manual step for configuring the necessary cookies, developers can load an existing browser profile into the chrome driver instance. I have experience using this approach with the gecko (firefox) selenium driver. I'm assuming it's pretty similar for chrome or other drivers.

The profiles/cookies that can be loaded is dependent on the browser a developer has installed on their machine and uses for browsing.

Selenium documentation for configuring the driver profile can be found here: https://www.selenium.dev/selenium/docs/api/py/webdriver_firefox/selenium.webdriver.firefox.firefox_profile.html

It's pretty simple to implement.

image

1220moritz commented 1 year ago

Will take a look. The scraper is currently broken so I'll try to fix it first. But sounds like a good suggestion, thanks!

krjii commented 1 year ago

I might have some time to throw something together. I have the initial code in place. I think the capability has many use cases. I haven't seen any other packages to support "authenticated" sessions through selenium.

I need about a week or 2. It's vacation time with family