mitmproxy backend problem with .exe

[P-Leo10]

Hi! I made an executable of my python program with seleniumwire where it opens a chrome webdriver with mitmproxy backend. Initially it was not going on the other pc that does not have python installed and I saw that the problem was that it did not have mitmproxy installed. I did the installation in mitmproxy-5.3.0 now. When it tries to open with backend, it gives the following error:

proxy returned status 502 for http://seleniumwire/initialise

I need an answer as soon as possible to solve my problem. Thank you already

[wkeeling]

Thanks for raising this. Does it work if you use the standard backend - as in, don't specify the backend option in the Selenium Wire options? Or is there a specific reason you need to use the mitmproxy backend?

[P-Leo10]

I thank you for the quick answer. So, I noticed that on the bet365 website, when I place live bets, seleniumwire cannot detect them because in the network they are not actually completed, even the bet365 server receives the bet. Using mitmproxy, he is able to capture live bets because the return goes as expected, but without using the mitmproxy backend it does not work. I'm hostage to the mitmproxy backend

[P-Leo10]

How does mitmproxy_confdir work? Can he possibly help with that?

[P-Leo10]

When I just do the executable and there is no mitmproxy installed on the pc, this is the error:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bet365.py", line 215, in abrir_chrome
  File "site-packages\seleniumwire\webdriver\browser.py", line 65, in __init__
  File "site-packages\seleniumwire\proxy\client.py", line 69, in create_proxy
  File "site-packages\seleniumwire\proxy\mitmproxy.py", line 46, in start
  File "subprocess.py", line 854, in __init__
  File "subprocess.py", line 1307, in _execute_child
FileNotFoundError: [WinError 2] O sistema não pode encontrar o arquivo especificado

please help me 😢

[P-Leo10]

that's why I need mitmproxy: https://github.com/wkeeling/selenium-wire/issues/170

[P-Leo10]

after cx_freeze:

in script C:\Users\menga\OneDrive\Desktop\Bet365\dist\lib\seleniumwire\proxy\mitmproxy.pyc: source code string cannot contain null bytes

@wkeeling @wkeeling

[wkeeling]

Are you trying to bundle the libraries into a single executable? The problem you will likely hit there is that selenium wire invokes mitmproxy (the mitmdump command to be exact) by running the command in a subprocess. That means that the command has to be available as a runnable script on the filesystem and won't work if it's embedded inside a .exe. Normally the mitmdump command would be available in the /bin folder of your virtual environment.

The alternative would be to use the standard (non-mitmproxy) backend and try and figure out why it isn't working with bet365. Are you seeing any errors in the console when you place live bets?

[P-Leo10]

Does it not have a way or file that I can place in the folder where the executable is created and make it look for that file the mitmdump?

About the problem with using seleniumwire without the mitmproxy backend is that I could see that the request does not return a status code, it is as if it were canceled, even if it is not. The error message is almost the same as the # 170. If you managed to fix this, I probably wouldn't have to use the backend, because I need the body of the request that I send in the browser, and even if the body of the request appears, because it is not actually completed in the browser, it does not capture it.

[wkeeling]

Are you able to post the exact trace back you're seeing?

[P-Leo10]

From when? after I make the executable or if I run without the mitmproxy backend? @wkeeling

[P-Leo10]

When I'm going to place a live bet at bet365 and it doesn't have the mitmproxy backend, it gives me this error:

Error making request
Traceback (most recent call last):
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\proxy\proxy2.py", line 92, in proxy_request
    res = conn.getresponse()
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 1347, in getresponse
    response.begin()
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\http\client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\socket.py", line 669, in readinto
    return self._sock.recv_into(b)
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

and the request on the chrome network, looks like this:

With that, the driver.wait_for_request() does not detect the request nor does it take the body data of the request itself, which is what I need to get. @wkeeling

[P-Leo10]

Using the mitmproxy backend, it works perfectly with my machine. The problem is that when I create the executable and pass it on to someone who doesn't have python and mitmproxy installed, it doesn't work. It gives the following error:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bet365.py", line 215, in abrir_chrome
  File "site-packages\seleniumwire\webdriver\browser.py", line 65, in __init__
  File "site-packages\seleniumwire\proxy\client.py", line 69, in create_proxy
  File "site-packages\seleniumwire\proxy\mitmproxy.py", line 46, in start
  File "subprocess.py", line 854, in __init__
  File "subprocess.py", line 1307, in _execute_child
FileNotFoundError: [WinError 2]  The system cannot find the file specified

When I install mitmproxy executable from windows on the mitmproxy website, the error becomes this:

in script C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\proxy\mitmproxy.py: No module named 'seleniumwire'
Traceback (most recent call last):
  File "C:\Users\menga\OneDrive\Desktop\Bet365\bet365.py", line 241, in abrir_chrome
    driver = webdriver.Chrome('./chromedriver', options=options, seleniumwire_options=backend)
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\webdriver\browser.py", line 65, in __init__
    addr, port = self._client.create_proxy(
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\proxy\client.py", line 79, in create_proxy
    self.initialise_proxy(options)
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\proxy\client.py", line 90, in initialise_proxy
    self._make_request('POST', '/initialise', data=options)
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\proxy\client.py", line 251, in _make_request
    raise ProxyException('Proxy returned status code {} for {}'.format(response.status, url))
seleniumwire.proxy.client.ProxyException: Proxy returned status code 502 for http://seleniumwire/initialise
Traceback (most recent call last):
  File "C:\Users\menga\OneDrive\Desktop\Bet365\bet365.py", line 241, in abrir_chrome
    driver = webdriver.Chrome('./chromedriver', options=options, seleniumwire_options=backend)
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\webdriver\browser.py", line 65, in __init__
    addr, port = self._client.create_proxy(
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\proxy\client.py", line 79, in create_proxy
    self.initialise_proxy(options)
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\proxy\client.py", line 90, in initialise_proxy
    self._make_request('POST', '/initialise', data=options)
  File "C:\Users\menga\AppData\Local\Programs\Python\Python38\lib\site-packages\seleniumwire\proxy\client.py", line 251, in _make_request
    raise ProxyException('Proxy returned status code {} for {}'.format(response.status, url))
seleniumwire.proxy.client.ProxyException: Proxy returned status code 502 for http://seleniumwire/initialise

@wkeeling

[P-Leo10]

I updated the previous post with the most complete error. @wkeeling

[wkeeling]

Ok thanks @P-Leo10

After you install mitmproxy can you open a Windows command prompt and run the command mitmdump? Does the command work, or do you get 'mitmdump' is not recognised as an internal or external command, operable program or batch file? If the mitmdump command is not found, you will need to add it to your PATH so that selenium wire can find it.

[P-Leo10]

Many thanks for the reply, @wkeeling.

I uninstalled mitmproxy by pip uninstall mitmproxy and installed the mitmproxy executable to simulate a pc that doesn't have python. That done, I ran mitmdump on cmd as requested and returned:

[wkeeling]

Are you still seeing the Proxy returned status code 502 error with the mitmproxy executable installed?

[P-Leo10]

sadly yes :cry: @wkeeling

[wkeeling]

The latest version of Selenium Wire (v4.2.0) has no external dependency on any executables. Confirmed that using cx_freeze to bundle a test script using Selenium Wire into a single .exe now works. I simply ran:

cx_freeze test.py --target-dir dist

And then I double-clicked dist\test.exe which started Chrome, visited a URL and captured a bunch of requests before ending