Certificates not valid on Chrome

[wkeeling]

Related to #2

This behavior has also been observed when using Chrome in a regular test. The certificate is not considered valid.

Windows Server 2016. Chrome version - 59.0.307 & latest

Test setup:

from seleniumwire import webdriver
from seleniumwire.webdriver import ChromeOptions
import time

options = {
    'https': 'proxy detail',
    'disable_encoding': True
}

browser = webdriver.Chrome(seleniumwire_options = options)
browser.header_overrides = { 'Cookie': 'cookie string' }

browser.get('url')

[wkeeling]

We need to modify the code that generates the server certificate:

https://github.com/wkeeling/selenium-wire/blob/faab19386447aabd7e64b354cb89929be62443c5/seleniumwire/proxy/proxy2.py#L95

... so that it includes the subjectAltName field. It seems that the way to currently generate this field is to use an ugly shell hack, specifically:

-config <(cat /etc/ssl/openssl.cnf \
    <(printf '[SAN]\nsubjectAltName=DNS:hostname,IP:192.168.0.1')) 

but this may not be possible or desirable in our case.

[djreducer]

Has there been any progress on this issue? Thanks

[wkeeling]

@djreducer the latest version of Selenium Wire (v1.0.5) does generate the subjectAltName field when it creates the server certificate. That means that if Selenium Wire's root certificate is installed into the browser, the browser shows the site as secure in the address bar when Selenium Wire is run in standalone mode (not remote controlling the browser - just proxying the traffic).

The problem I have currently is that after installing the root certificate into the browser, it only seems to take effect for regular (non-remote controlled) browser instances. When the browser is remote controlled by Selenium Wire, then for some reason the previously installed root certificate is not listed in the certificate authorities section, and the site still displays as "insecure".

So I'm still trying to figure that out. Perhaps it's some kind of "profile" issue (root certificate available in one profile but not the other)? If you have any insights yourself let me know!

The root certificate can be obtained by running:

python -m seleniumwire extractcert  # Will produce ca.crt

[JasonCrowe]

Does the site that selenium visits know that there is an issue with certificates, or is this only a browser facing problem?

[wkeeling]

@JasonCrowe it's something that exists between the browser and Selenium Wire itself. The end site is none the wiser.

[wkeeling]

Using the latest version of Google Chrome with Selenium Wire's root certificate installed does not appear to have this problem. Chrome shows the padlock icon next to the site being visited.

To install the root certificate:

• Extract the certificate by running python -m seleniumwire extractcert. That will produce a file called ca.crt in the current working directory.
• Open Chrome's settings and search for "Manage certificates".
• On the Manage certificates screen, click the "Authorities" tab and then "Import".
• Select the ca.crt file and ensure that you check all boxes relating to trust settings before clicking "OK".

[Ort3xx]

Hi, got this Problem whit the Certificate, when I use python -m seleniumwire extract cert there will be no file created

[wkeeling]

Do you see an error? You need to make sure that you run that command in the virtual env in which Selenium Wire is installed (assuming you're using a virtual env).

Alternatively, you can download the file from here:

https://raw.githubusercontent.com/wkeeling/selenium-wire/master/seleniumwire/proxy/ca.crt

[Ort3xx]

I'm very new to this, can you discribe it for noobs 😅 I creat a script to view websites( traffic) and there a some videos got counted, I think they don't autoplay back because of the failing cert. When I open the site in normal browser it works

[wkeeling]

hmm... not sure the invalid certificate error would prevent the videos from autoplaying - may be some other issue.

Do you have an example URL I can use to reproduce?

[Sajzad]

stilling facing not secure issue post installation of root certificate.

[TheBestMoshe]

To install the root certificate:

Extract the certificate by running python -m seleniumwire extractcert. That will produce a file called ca.crt in the current working directory. Open Chrome's settings and search for "Manage certificates". On the Manage certificates screen, click the "Authorities" tab and then "Import". Select the ca.crt file and ensure that you check all boxes relating to trust settings before clicking "OK".

Any way to do this programmatically? I want to set this up in a headless Chrome Docker container

[wkeeling]

@TheBestMoshe I'm afraid I don't know of a way to do it programmatically. However, one workaround could be to install the certificate manually into a Chrome profile (e.g. create a profile in Chrome on your local workstation), copy the profile into the Docker container, and then tell Chrome to use that profile when you create the webdriver instance - e.g.

from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("user-data-dir=/path/to/profile/with/cert/inside/container")
driver = webdriver.Chrome(chrome_options=chrome_options)

Sorry I can't give you a better solution.

There's more info on using Chrome profiles with Selenium here and here

[wkeeling]

The root certificate was using an out of date hashing algorithm which was causing Chrome to flag it as insecure. The certificate has now been updated (available in version 1.2.1 or higher of Selenium Wire).

I will close this ticket for now, but can re-open if certificate issues on Chrome continue to occur.

[Bishwas-py]

I am getting the same issue!

[mamparra]

I am getting the same issue!

Just managed to fix it by importing the cert as described by wkeeling above. It required importing into Trusted Root Certification, rather than the Authorities referenced above.

To install the root certificate:

Extract the certificate by running python -m seleniumwire extractcert. That will produce a file called ca.crt in the current working directory.
Open Chrome's settings and search for "Manage certificates".
On the Manage certificates screen, click the "Authorities" tab and then "Import".
Select the ca.crt file and ensure that you check all boxes relating to trust settings before clicking "OK".

[wkeeling]

Thanks @mamparra

It looks as though the naming may be different depending on platform. On Chrome 90 on Linux the tab is called "Authorities". Are you on Windows?

[mamparra]

Wow, ninja quick!

I am yes. Server 2016. This is the version of Chrome: Version 90.0.4430.212 (Official Build) (64-bit)

Apologies if i've hijacked a linux thread...

[wkeeling]

Thanks @mamparra ! I'll update the doc so it reflects both Windows and Linux.

[Anandesh-Sharma]

How would I do achieve this, if I am on a server with no GUI, how would I add certificates then, any ideas ?

[Bool13]

502 Bad Gateway Certificate verification error for update.googleapis.com: self signed certificate in certificate chain (errno: 19, depth: 1)

The certificate is displayed as valid. There is an error in the code 127.0.0.1:1038: Invalid certificate, closing connection. Pass --ssl-insecure to disable validation. 127.0.0.1:1038: Certificate verification error for update.googleapis.com: self signed certificate in certificate chain (errno: 19, depth: 1)

seleniumwire_options = { 'connection_timeout': None, 'proxy': { 'http': 'http://127.0.0.1:8080', 'https': 'http://127.0.0.1:8080', 'no_proxy': 'localhost, 127.0.0.1' }, 'verify_ssl': False }

options = webdriver.ChromeOptions () options.add_argument ('- ignore-certificate-errors')

chrome = webdriver.Chrome (r './ chromedriver.exe', chrome_options = options, seleniumwire_options = seleniumwire_options)

I use burp for proxying.

[wkeeling]

@Anandesh-Sharma if you're on Linux you can use the certutil command to install the certificate from the command line:

certutil -d sql:$HOME/.pki/nssdb -A -t TC -n "Selenium Wire" -i /path/to/ca.crt

[taylornelsonbluecargo]

Make sure to initialize the db first certutil -d $HOME/.pki/nssdb -N