" Not Secure " while browsing with selenium-wire

[hobocow]

On some websites, requests cannot be made through the browser because the connection is not considered as a secure connection.

I tried installing the certificate on my computer (mac) and tried to start selenium wire with a local chrome profile too, still not working.

Same on firefox.

Any workaround ?

[wkeeling]

Thanks for raising this. What does the browser say on websites that it can't connect to - do you get a specific error message?

[hobocow]

Hi,

It's not so much about the browser itself, i can browse any websites as I would on a regular browser / selenium. But for example if I try to login on some websites, the request wouldn't get through.

An example would be this : https://www.nike.com/login

Working fine on regular browser/seleniuim but in selenium wire it's just an unknown error systematically.

I think its because in selenium wire there is always the " not secure " message at the top left corner of the browser ? In regular selenium i dont have that 👍

[wkeeling]

Ok thanks. The "not secure" indicator in the browser address bar indicates the browser doesn't trust Selenium Wire's root certificate, but this wouldn't normally stop you from accessing or logging into a site.

I'll see if I can reproduce with nike.com.

[hobocow]

Thank you, that is what i thought too.

In the mean time is there any way to make the browser trust selenium wire's certificate ?

[wkeeling]

Have you manually installed the ca.crt into Chrome? If you do that, Chrome should then trust the certificate and the "not secure" indicator will be replaced with a padlock icon.

To get the ca.crt you can run:

python -m seleniumwire extractcert

Or alternatively download it from here: https://raw.githubusercontent.com/wkeeling/selenium-wire/master/seleniumwire/proxy/ca.crt

[hobocow]

Yes I have tried installing it on both mac and windows, however, im not sure what you mean by install it into chrome ?

I only installed the certificate in both system : in the key chain access app on mac os and just double clicking on windows, that should do the job ? I couldnt find anyway to install a certificate directly in chrome

edit : i managed to install it directly in my chrome in windows and managed to login ! thank you so much ! however i cant see the post requests it makes with

for request in driver.requests: if request.response: print( request.path, request.response.status_code, request.response.headers['Content-Type'] )

edit 2 : it couldnt get all the requests because ome requests dont have content-type and it made the script stop, problem solved and everything is working perfectly now, thank you so much for your time and your work !!!!! 😄

[wkeeling]

Great, glad to hear it's working!

[P-Leo10]

@hobocow @wkeeling Hi bros, I have the same problem to access https://www.nike.com.br. Could you tell me how to install ca.crt for windows? I do not even know where to begin with. I´m using Chrome and the new version of Selenium Wire (v2.00)

edit1: I got it thanks to this answer https://github.com/wkeeling/selenium-wire/issues/97#issuecomment-605614701

[wkeeling]

@P-Leo10 The root certificate is here: https://raw.githubusercontent.com/wkeeling/selenium-wire/master/seleniumwire/proxy/ca.crt

Copy all the text (including the BEGIN and END lines) and save it in a new file called ca.crt. Then go to Chrome's settings, then to "Certificates" (you can search the settings for "Certificates"). From there locate the "Certificate Authorities" section and use the "Import" function to import the ca.crt file you just created.

[P-Leo10]

@wkeeling thank you very much for the help, it worked. The only problems that have not yet come out are: -this error, client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101 -this error in the browser, runned with selenium-wire ERR_PROXY_CONNECTION_FAILED after the script execution

[wkeeling]

@P-Leo10 could you post the code that you're using to create the webdriver and the full error traceback you're seeing?

[P-Leo10]

@wkeeling I'm working with a larger and more complex code but I did a test with this simple code and even in this script it informs me this error in the terminal

code:

import requests
from seleniumwire import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.chrome.options import Options

driver = webdriver.Chrome('./chromedriver')
driver.get('https://www.nike.com.br/')

error:

[86924:97464:0721/051946.790:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.851:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.855:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.856:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.899:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.901:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.917:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.961:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.963:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051946.977:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101
[86924:97464:0721/051947.023:ERROR:ssl_client_socket_impl.cc(959)] handshake failed; returned -1, SSL error code 1, net_error -101

moreover, thanks to your Selenium-wire, I am trying to create a way to overcome recaptcha v3, if you can give me a means to communicate with you in a non-public way, it would be great.

[wkeeling]

It might be worth switching the backend to see if that makes a difference. Try that first, but happy to continue the conversation non-publicly if no joy.

[P-Leo10]

@wkeeling i try with

options  = {
     'backend' : 'mitmproxy'
}

but when i used this, the problem continues and comes back the problem with 'Not secure'. I expressed myself wrong, I wanted to speak to you in private so as not to publish my idea publicly, I'm using the google translator so I don't know if I'm writing 100% right hahaha

[wkeeling]

With switching the backend to mitmproxy, you will need to ensure that you install the mitmproxy root certificate (it is different to the ca.crt that you already installed). Once installed, that should make the "Not secure" message go away. You can find instructions on installing the mitmproxy certificate here: https://docs.mitmproxy.org/stable/concepts-certificates/#installing-the-mitmproxy-ca-certificate-manually

[shanebenlolo]

@P-Leo10 The root certificate is here: https://raw.githubusercontent.com/wkeeling/selenium-wire/master/seleniumwire/proxy/ca.crt

Copy all the text (including the BEGIN and END lines) and save it in a new file called ca.crt. Then go to Chrome's settings, then to "Certificates" (you can search the settings for "Certificates"). From there locate the "Certificate Authorities" section and use the "Import" function to import the ca.crt file you just created.

Hey wkeeling, I tried to follow this link, as I am having the same problem as the original poster and am trying to work the issue out by adding the root certificate to chrome, but the link now appears to be dead. Do you have a working link to this file anymore? Thanks so much

[wkeeling]

@shanebenlolo Thanks for the message. The certificate has moved since that comment containing the link was posted. You should be able to download it here: https://github.com/wkeeling/selenium-wire/raw/master/seleniumwire/ca.crt (also accessible from the Certificates section in the doc).

You can also get the certificate by running the following command:

python -m seleniumwire extractcert

[KarateKugler]

It's important to import the certificate in the "Trusted Certificates" Tab.

[Esaanjum]

On some websites, requests cannot be made through the browser because the connection is not considered as a secure connection.

I tried installing the certificate on my computer (mac) and tried to start selenium wire with a local chrome profile too, still not working.

Same on firefox.

Any workaround ?

This is the only solution I know of: use selenium webdriver instead of seleniumwire webdriver

[wkeeling]

If you're using a Linux based system the certificate can also be installed from the terminal using the following command:

certutil -d sql:$HOME/.pki/nssdb -A -t TC -n "Selenium Wire" -i /path/to/ca.crt

Change /path/to/ca.crt appropriately.

If you get a "command not found" then certutil can be installed with:

apt install libnss3-tools

[reeemsalah]

It's important to import the certificate in the "Trusted Certificates" Tab.

@KarateKugler I can't find that tab inside google chrome settings, I can only find "Trusted People" and "Trusted Publishers". I have tried importing the certificate to both tabs but still, when I try to open any website using selenium-wire, it fails to find a certificate for it.

[wkeeling]

@reeemsalah there should be a tab that looks like the following: If you don't have that, it's possible it may have been disabled by your organisation - e.g. by a company-wide security policy.

[reeemsalah]

@wkeeling Thank you so much, it worked 💯

[yeyuchen198]

seleniumwire和undetected_chromedriver，证书不安全警告问题！

解决办法，新开一个CMD窗口，输入命令： python -m seleniumwire extractcert 就会在同级文件夹生成一个ca.crt证书文件，然后在Chrome浏览器 > 设置里面搜索证书，管理证书，在【受信任的根证书颁发机构】那一栏，导入ca.crt这个证书！这样就没有证书不安全的警告了！！

[cheezypotatoes]

@wkeeling should I continue?

[cheezypotatoes]

nvm, I realize you own this package

[omid-adibfar]

@P-Leo10 The root certificate is here: https://raw.githubusercontent.com/wkeeling/selenium-wire/master/seleniumwire/proxy/ca.crt

Copy all the text (including the BEGIN and END lines) and save it in a new file called ca.crt. Then go to Chrome's settings, then to "Certificates" (you can search the settings for "Certificates"). From there locate the "Certificate Authorities" section and use the "Import" function to import the ca.crt file you just created.

As an update, the link is deprecated, you can download ca.crt here: https://raw.githubusercontent.com/wkeeling/selenium-wire/master/seleniumwire/ca.crt

[somT-oss]

@wkeeling I tried your solution for Linux-based systems, but it did not work.

Is there any workaround?

[A4Abhiraj]

Try this: This should extract the certificate in the dir you run this command from. python -m seleniumwire extractcert

Open your chrome, go to settings > privacy & security > security > manage device certificates > authorities > click import select the ca.crt that the previous command had generated.

[Act0r1]

@P-Leo10 The root certificate is here: https://raw.githubusercontent.com/wkeeling/selenium-wire/master/seleniumwire/proxy/ca.crt

Copy all the text (including the BEGIN and END lines) and save it in a new file called ca.crt. Then go to Chrome's settings, then to "Certificates" (you can search the settings for "Certificates"). From there locate the "Certificate Authorities" section and use the "Import" function to import the ca.crt file you just created.

when i try to add it, i get error: The Private Key for this Client Certificate is missing or invalid

[ravigadila]

@Act0r1 same issue Got "The Private Key for this Client Certificate is missing or invalid" when trying to import ca.crt

[Lovi-0]

Doesnt work -> Client certificate is missing or invalid

[unchaineddev]

Are you adding it in the “Trusted Root Certification Authority" section? @Ghost6446 Try this

[Act0r1]

@Act0r1 same issue

Got "The Private Key for this Client Certificate is missing or invalid"

when trying to import ca.crt

First, generate the certificate - python -m seleniumwire extractcert then install it - certutil -d sql:$HOME/.pki/nssdb -A -t TC -n "Selenium Wire" -i /path/to/ca.crt

Maybe it will help someone And I highly recommend just use playwrig, there no error like this

[milahu]

summary:

• the ca.crt file must be imported as an *authority* certificate:
  chrome > settings > security > manage device certificates > authorities > import
• the certificate is stored in $HOME/.pki/nssdb/cert9.db and $HOME/.pki/nssdb/key4.db.
  it is not stored in chromium's user-data-dir
• adding the certificate can be automated with certutil from nss-tools:
  mkdir -p $HOME/.pki/nssdb; certutil -d sql:$HOME/.pki/nssdb -A -t TC -n "Selenium Wire" -i /path/to/ca.crt

i made nssdb-py to replace certutil -A to add a ca.crt certificate to $HOME/.pki/nssdb/cert9.db

problem is: cloudflare will block the https proxy with self-signed certificate see also #656

[premrakh-vision]

the ca.crt file must be imported as an authority certificate: chrome > settings > security > manage device certificates > authorities > import

when I follow above step to handle Not Secure protocol its work but these setting do manually I want to fix this issue via coding if possible so please answer it

[milahu]

adding the certificate can be automated with certutil