search

wkeeling / selenium-wire

Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.
MIT License
1.9k stars 254 forks source link

Selenium-wire certificate is not valid #137

Closed MrFzovpec closed 2 years ago

MrFzovpec commented 4 years ago

I'm using Chrome 84, selenium-wire 2.0.0, python 3.7, Mac OS Catalina I need to connect to the website with https connection, but Chrome tells me, that certificate is not trusted (not valid) and changes https connection to http. I had tried all ways from #31 , but there're no results yet. Could anyone help me?

This is my simple code example

from seleniumwire import webdriver import time

browser = webdriver.Chrome(executable_path='./drivers/chromedriver') browser.get('https://google.com') time.sleep(20) browser.close()

Results in Chrome (In Russian, sorry)

Снимок экрана 2020-07-21 в 11 54 34 Снимок экрана 2020-07-21 в 11 54 44
wkeeling commented 4 years ago

Does the certificate info give any indication as to why it thinks the certificate is not valid?

I've just updated Chrome to 84 and tried reproducing, but Chrome indicates that the site is secure. I used https://www.google.com however to prevent a redirect when the www is omitted. Can you see if that makes any difference?

MrFzovpec commented 4 years ago

Does the certificate info give any indication as to why it thinks the certificate is not valid?

I've just updated Chrome to 84 and tried reproducing, but Chrome indicates that the site is secure. I used https://www.google.com however to prevent a redirect when the www is omitted. Can you see if that makes any difference?

I deleted certificate from the macOS keychain to find out the issue, but Chrome tells me just "Certificate of www.google.com is not trusted" (Red text in Russian)

I also added www to the url, but there was no difference.

Снимок экрана 2020-07-21 в 12 17 29
wkeeling commented 4 years ago

Thanks. Can you try downloading and reinstalling the root certificate? The certificate is here: https://github.com/wkeeling/selenium-wire/raw/master/seleniumwire/proxy/ca.crt

A fresh install of the latest certificate may resolve perhaps.

MrFzovpec commented 4 years ago

@wkeeling, mac os keychain also tells me, that this certificate (new) is not trusted. Maybe there is a problem with certificate only in Mac OS?

Снимок экрана 2020-07-21 в 12 26 06
wkeeling commented 4 years ago

Hmmm... that's weird. Perhaps it is a Mac OS specific issue with the certificate as you say. I don't have a Mac handy so I'm not able to reproduce (I was using Linux for my previous test above).

It might be worth switching the backend to see whether that allows you to move forward. The mitmproxy backend uses a different root certificate.

MrFzovpec commented 4 years ago

Hmmm... that's weird. Perhaps it is a Mac OS specific issue with the certificate as you say. I don't have a Mac handy so I'm not able to reproduce (I was using Linux for my previous test above).

It might be worth switching the backend to see whether that allows you to move forward. The mitmproxy backend uses a different root certificate.

Won't be any problems if I'll use my custom proxies?

MrFzovpec commented 4 years ago

So, there is the same problem, but with mitmproxy certificate. I'll try to fix it.

Снимок экрана 2020-07-21 в 12 37 09
MrFzovpec commented 4 years ago

@wkeeling is it possible to use another certificate or default system certificate (I suppose, like in common selenium)? Because a problem with certificates (mitmproxy and ca.crt) hasn't been solved.

wkeeling commented 4 years ago

@MrFzovpec I do not know of a way to do that unfortunately. Have you tried using an alternative browser - e.g. Firefox?

vikingliu commented 4 years ago

I face the same problem in mac os. url = https://market.m.taobao.com/app/tmall-def/jhsbybt/web/index.html works in selenium, not valid in selenium-wire. if you can fix this, it's great help

adrianbeloqui commented 4 years ago

+1

soumyaevan commented 3 years ago

I am facing the same issue in mac OS. when opening a site using webdriver from selenium-wire, it is getting not secured

masudr4n4 commented 3 years ago

image I also got the same issue in windows 10.

wkeeling commented 3 years ago

@masudr4n4 Have you installed Selenium Wire's root certificate as an authority?

masudr4n4 commented 3 years ago

@wkeeling I am not sure what are you talking about i just installed seleniumwire via pip...How to do that?It does not show me the error if its only selenium

wkeeling commented 3 years ago

@masudr4n4 Yes it won't show you the error if you're just using Selenium, because Selenium does not intercept and capture requests.

As explained in the certificates section, if you wish to make the "Not Secure" message disappear, then you'll need to install Selenium Wire's SSL certificate as an authority on your computer. You can grab the certificate here. Save it as to a file called ca.crt, and then go to your browser settings, find the certificates section, and install the certificate in the "authorities" section.

Selenium Wire will function without the certificate being installed, but installing it will ensure that you get a padlock icon in the address bar.

ofafa commented 2 years ago

I found it could be fixed by changing the configuration in keychain like below:

Screen Shot 2022-01-14 at 11 36 04 PM
  1. go to keychain and find the certificate
  2. double click on it
  3. click on the right arrow of Trust section
  4. adjust the trust settings according your scenario

In my case, I need to have https connection to the site I'm scraping to work, so I need to enable SSL, other setting are not important.

wkeeling commented 2 years ago

Closing this issue for now.

cwazuidema commented 1 year ago

@MrFzovpec, can you please edit your top post and add @ofafa answer directly in your post? It will help devs to find the answers directly. Thanks

prachib-vision commented 9 months ago

i don't want my client to do this manually, is there any way to do it in programmatic way in seleniumwire package only???