search

Vasistareddy / any_pagesource_scraper_selenium

Extract pagesource/html from the selenium webdriver with get request of Flask API
2 stars 2 forks source link

Can't seem to use proxies on this configuration #1

Open gerimo opened 5 years ago

gerimo commented 5 years ago

Hi Vasista!

Thank you for building this. You are awesome.

I've been trying all day to set up a proxy for this, and I can't seem to find a way around it:

                     ```
                     proxy = "178.20.231.218"
        proxy_port = 80
        url = "https://www.whatsmyip.org/"

        fp = webdriver.FirefoxProfile()
        # Direct = 0, Manual = 1, PAC = 2, AUTODETECT = 4, SYSTEM = 5
        fp.set_preference("network.proxy.type", 1)
        fp.set_preference("network.proxy.http",proxy)
        fp.set_preference("network.proxy.http_port",proxy_port)
        fp.update_preferences()

                     #where this URL returns the proxy ip
        driver = webdriver.Firefox(firefox_profile=fp)
        driver.get(url)
                     ```

Request are still sent from my home ip address.

Have you find a way around this?

Vasistareddy commented 5 years ago

Hey @gerimo

Thanks for the compliments. Have you find a way on this? Sorry for replying late.

I have commented two lines in app.py, just un-comment them and fill the proxy_ip and proxy_port to test.

# fp.set_preference("network.proxy.http", "<proxy ip>")
# fp.set_preference("network.proxy.http_port", "<proxy port")

These lines worked for me. Feel free to discuss.

Thanks