When redistributing python exe, seleniumwire ca.crt not found in temp folder

[stjkr]

On the machine that I coded on, the chrome browser opens by selenium using the chromedriver. When I convert code to .exe it still runs the same as it would in the IDE.

When I use the .exe on another machine which still has python installed but not selenium/wire, then it runs into this problem when trying to open.

[Errno 2] No such file or directory: 'C:\\Users\\Owner\\AppData\\Local\\Temp\\_MEI123722\\seleniumwire\\ca.crt'

After manually putting the certificate (ca.crt) into the folder and run again, I get this problem. _Note that I'm using a tkinter button to start the browser so that's why the _MEI123722 doesn't change. If I did reload the program then it changes to something like _MEI123801_

[Errno 2] No such file or directory: 'C:\\Users\\Owner\\AppData\\Local\\Temp\\_MEI123722\\seleniumwire\\ca.key'

How can I fix this so I can make my software distributable.

I'm running Windows on both machines, using python 3.9(64).

[stjkr]

@wkeeling I'm quite new to github but have been coding for a while, but I don't know how to DM you with this problem. It's kind of urgent though. Sorry about the tag.

[wkeeling]

Thanks for raising this.

Can you show how you're building the .exe? Are you using pyinstaller?

[stjkr]

@wkeeling I use this pyinstaller --onefile -i"logo.ico" script.py

[wkeeling]

OK thanks. I can reproduce the issue, although I don't have a solution yet. It appears that the ca.key and ca.crt files need to be supplied as data files to pyinstaller. I followed the instructions in the link, but I also receive the error [Errno 2] No such file or directory: '/tmp/_MEICcMA6m/seleniumwire/ca.crt'. When I get some more time I'll dig a bit deeper.

[stjkr]

Thanks a lot, I've got a tonne of code I'm trying to distribute but this is the 1 thing that is stopping me lol :( I've looked everywhere and have no clue. I tried getting rid of the ca.crt from the browser and it didn't work. Just made every site "not secure". Thanks for your response though

[wkeeling]

There is one thing worth trying in the short term. Try making a file called seleniumwire-ca.pem which contains ca.crt and ca.key - basically just concatenate these two files into a single file called seleniumwire-ca.pem. Place that single concatenated file in C:\\Users\\Owner\\AppData\\Local\\Temp\\.seleniumwire (you may have to make the folder .seleniumwire if it doesn't exist).

[stjkr]

I have noticed that there is seleniumwire-ca.pem and seleniumwire.dhparam.pem in C:\Users\Owner\.seleniumwire

I put both of those files in C:\\Users\\Owner\\AppData\\Local\\Temp\\.seleniumwire which was already a folder. The problem is I don't know how I could reproduce this. I don't want my customers having to put that in their file. Is there a way to do this automatically?

Thank you though!

[stjkr]

@wkeeling is it safe to distribute the .pem file to other people. When I take a closer look it says: BEGIN PRIVATE KEY. Will this work on other peoples devices and is it safe for me to distribute it?

[wkeeling]

The key is what Selenium Wire uses to decrypt/encrypt data for interception. It's fine to distribute (it's distributed as part of the library anyway).

[stjkr]

Okay, it's just it said private key in the .pem file so I was confused, thank you.

Is there anyway for my users to automatically have it installed.

[wkeeling]

There's no need for users to install the key - users would only normally install ca.crt if they wished to get rid of the insecure icon displayed by the browser. However, Selenium Wire does need access to both the key and crt to perform request interception.

The way it works normally is Selenium Wire loads ca.crt and ca.key from its library module, concatenates them to a single file called seleniumwire-ca.pem and stores that in <TEMP>/.seleniumwire. From that point on, Selenium Wire will just use seleniumwire-ca.pem directly. It will only try and load ca.crt and ca.key again if seleniumwire-ca.pem disappears (e.g. is deleted).

It still feels like we need to crack the issue with PyInstaller not properly providing ca.crt and ca.key as this will resolve the issue properly. I'll try and do that as soon as I have some proper time.

One final thing to mention is that you can control the location of the .seleniumwire folder with the request_storage_base_dir option. That defaults to the system TEMP folder, but if you wanted to change that you can control it using that option.

[stjkr]

Wow, thanks so much!

I've used the 'request_storage_base_dir' option so now it creates a temp folder within the main directory. I think it's solved now.

Only problem is a tonne of data is being stored in .seleniumwire but not deleted.

[wkeeling]

Ok that's good news. In terms of the request data, Selenium Wire will periodically clean out the data itself, but you can also do that manually with del driver.requests from within your code. Alternatively you can limit what gets stored by using scopes (see "Limiting Request Capture" in the README). There's also the option of using memory only storage so nothing gets stored on disk (see "In-Memory Storage" in the README). So maybe one of these mechanisms will help with that.

[stjkr]

That's great. Thanks so much for your help! Life saver.

[waslost0]

Same issue, but i do not have any seleniumwire-ca.pem or ca.crt. I added files to C:\Users\me\AppData\Local\Temp\.seleniumwire\, but nothing changed

[Vegoo89]

First, as mentioned above, set your selenium wire storage director. Example:

selenium_wire_storage = os.path.join(
    os.getcwd(), "selenium_wire")

selenium_wire_options = {
    'request_storage_base_dir': selenium_wire_storage
}

and include it in webdriver initialization

driver = webdriver.Chrome(seleniumwire_options=self.selenium_wire_options)

Simple - but a bit dirty - workaround for the problem is creating in your project directory script called fe. certs.py and putting your certs there as dictionary. It removes the need of adding certs as data to pyinstaller, which can be tedious and for me it was still producing problems.

So in certs.py I created dict called certs_dict with two .pem files that I found after first run of my script (using python interpreter, not pyinstaller compiled binary).

Then you can create a function in your chromedriver initialization class. My function looks like this:

    def _copy_certs(self):

        destination_dir = os.path.join(
            self.selenium_wire_storage,
            ".seleniumwire")

        os.makedirs(name=destination_dir, exist_ok=True)

        for file_name, cert_string in certs_dict.items():

            destination_path = os.path.join(
                destination_dir, file_name)

            with open(destination_path, "w") as destination_cert_file:
                destination_cert_file.write(cert_string)

            logging.info(f"Cert copied to {destination_path}!")

I am calling it after I define my selenium_wire_storage class variable and before chromedriver initialization. I know it's far from perfect but as I said, it is easy workaround.

[champixn]

Wow, thanks so much!

I've used the 'request_storage_base_dir' option so now it creates a temp folder within the main directory. I think it's solved now.

Only problem is a tonne of data is being stored in .seleniumwire but not deleted.

How did you solve the issue?

[stjkr]

@champixn the data will be deleted automatically after a certain amount of time. You can manually add an option in the code to automatically do it though. The code is somewhere here I'm not sure where though, I forgot.

[champixn]

sorry, I mean how did you fix the main issue of missing file cz I'm a little confused

[stjkr]

@champixn what do you mean by missing file. Python tends to create a file when you write to it if it's not there.

[ribenz]

@wkeeling I have the same issue here, I tried to add the request_storage_base_dir option, and it does create a '.seleniumwire' folder in the place specified, but the ca.cert file and etc. aren't there, and seleniumwire still looks for the certificate files in the temp folder and fails to do so, because they're not there of course.

[gordianbahri]

@fullyHouse You have to create them like @wkeeling explained above if they don't exist.

[gordianbahri]

@wkeeling In my case, request_storage_base_dir is now pointing to my distribution folder and a .seleniumwire folder was created successfully. I added the seleniumwire-ca.pem to it and on my computer, it works like a charm. However, when copying the distribution to my Laptop and using it I'm facing the same problem again of displaying the insecure connection message on the browser.

My program dynamically catches the current path of the distribution folder and sets request_storage_base_dir on every start based on that value for each individual device it's running on, so seleniumwire not finding the '.seleniumwire' shouldn't be the problem, as it also isn't creating any .seleniumwire folder outside of the distribution folder. In addition, on both devices selenium stores, all new requests in the '.seleniumwire' folder that is inside my distribution folder, and deleting the seleniumwire-ca.pem will result in the browser window not opening, so it really seems like that seleniumwire is detecting my '.seleniumwire' folder and the seleniumwire-ca.pem in it, but only on one device it's showing a secure connection in the browser and on the other device it isn't. However, as I said both devices require seleniumwire-ca.pem to make selenium-wire execute its task. I also deleted seleniumwire-ca.pem again and put ca.crt and ca.key manually inside of the .seleniumwire folder on each device which also didn't work for me.

What I've also recognized was that selenium seems to store a ca.crt and ca.key in C:\\Users\\Owner\\AppData\\Local\\Temp\\_MEI.....\\site-packages\\seleniumwire on both devices.

As my program always downloads the newest version of chromedriver in the cache the solution of @Vegoo89 is also not practicable for me as it requires manual changes to a chromedriver class.

On my side, I spent a lot of time trying to resolve this issue but haven't come up with a solution yet. I know that the insecure connection message is only a local thing on the machine, but it like to fix this anyways. As this is also kind of urgent to me, I'm now posting this comment. Thanks in advance.

[wkeeling]

[Errno 2] No such file or directory: 'C:\\Users\\Owner\\AppData\\Local\\Temp\\_MEI123722\\seleniumwire\\ca.key' How can I fix this so I can make my software distributable.

To resolve this error it's necessary to tell pyinstaller about the two certificate files using the --add-data option when you compile the binary. This allows pyinstaller to handle the calls made by Selenium Wire when Selenium Wire first loads the certificates. (Selenium Wire uses the standard mechanism of pkgutil.get_data() to load these files internally.)

Steps:

• Create the two certificate files ca.crt and ca.key in the current working directory.
• Specify these files using --add-data when you run pyinstaller - for example:

  pyinstaller --add-data 'ca.crt;seleniumwire' --add-data 'ca.key;seleniumwire' --onefile my_prog.py

  (where my_prog.py is the name of your script - change as appropriate.)

You can discard the two local certificate files once the binary has been built.

If running on Mac/Linux, use a : for the separator, e.g. --add-data 'ca.crt:seleniumwire'

[therealkiwi66]

[Errno 2] No such file or directory: 'C:\\Users\\Owner\\AppData\\Local\\Temp\\_MEI123722\\seleniumwire\\ca.key' How can I fix this so I can make my software distributable.

To resolve this error it's necessary to tell pyinstaller about the two certificate files using the --add-data option when you compile the binary. This allows pyinstaller to handle the calls made by Selenium Wire when Selenium Wire first loads the certificates. (Selenium Wire uses the standard mechanism of pkgutil.get_data() to load these files internally.)

Steps:

• Create the two certificate files ca.crt and ca.key in the current working directory.
• Specify these files using --add-data when you run pyinstaller - for example:

pyinstaller --add-data 'ca.crt;seleniumwire' --add-data 'ca.key;seleniumwire' --onefile my_prog.py

(where my_prog.py is the name of your script - change as appropriate.)

You can discard the two local certificate files once the binary has been built.

If running on Mac/Linux, use a : for the path, e.g. --add-data 'ca.crt:seleniumwire'

thank you very much

[mynoob2]

I have used that code and I get an error Unable to find "C:\test compile\'ca.crt" when adding binary and data files when I have both the ca.crt and ca.key in the same directory as the file I am using pyinstaller on. Any suggestions?

[mynoob2]

I solved this issue by

1. Placing ca.crt and ca.key in C:\Users\OWNER\AppData\Local\Temp.seleniumwire
2. Running my .py file
3. Have ca.key and ca.crt in the working directory
4. Compiling with pyinstaller using --add-data 'ca.crt;seleniumwire' --add-data 'ca.key;seleniumwire' --onefile my_prog.py

[mynoob2]

I started getting the same issue again when trying to compile. Oof down the rabbit hole I go again following those steps.... hmm

[mynoob2]

Tried compiling on another computer and still no luck. Throws the same error, not being able to find the Ca.crt file. The script runs no problem so not sure why I am getting this issue.

[mynoob2]

Solved the issue lol. The proper script for windows is pyinstaller --add-data ca.crt;seleniumwire --add-data ca.key;seleniumwire --onefile pythonprogram.py does not work with the ' added

[lovepeacee]

Solved the issue lol. The proper script for windows is pyinstaller --add-data ca.crt;seleniumwire --add-data ca.key;seleniumwire --onefile does not work with the ' added

Where should I go to find ca.crt and ca.key? please！！！

[mynoob2]

Solved the issue lol. The proper script for windows is pyinstaller --add-data ca.crt;seleniumwire --add-data ca.key;seleniumwire --onefile does not work with the ' added

Where should I go to find ca.crt and ca.key? please！！！

Can get them from the GitHub or from your temp folder selenium wire in appdata

[hasham56]

Another workaround would be to add ca.crt and ca.key in your working directory, and then using 'ca_key': f'{os.getcwd()}\\ca.key' and 'ca_cert': f'{os.getcwd()}\\ca.crt' options in your seleniumwire_options.