wkeeling / selenium-wire

Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.
MIT License
1.9k stars 254 forks source link

Selenium wire not working with mitm proxy #49

Closed appunni-m closed 3 years ago

appunni-m commented 5 years ago

selenium-wire==0.10.0 mitm==4.0.4 Selenium wire is giving 400 (invalid-http-request-form-expected-authority-or-absolute-got-relative) with mitm in upstream mode. To run mitm in upstream proxy mode use this. mitmproxy -p 3129 --mode upstream:"http://upstream:port"

appunni-m commented 5 years ago

I have already tested the proxy with Google Chrome and python requests both of which didn't give any issue.

wkeeling commented 5 years ago

Thanks @appunni-dishq for raising. I'll see if I can reproduce. Can you post the selenium wire config that you're passing to the webdriver?

appunni-m commented 5 years ago
from seleniumwire import webdriver

proxy = {
            'http': settings.ANALYTICA_HTTP_PROXY,
            'https': settings.ANALYTICA_HTTPS_PROXY,
            'no_proxy': 'localhost,127.0.0.1'
            }
options = webdriver.ChromeOptions()
options.add_argument('--headless')
prefs = {
   "profile.managed_default_content_settings.images": 2,
   "profile.default_content_settings.images": 2
 }
options.add_experimental_option("prefs", prefs)
options.add_argument("--disable-application-cache")
options.add_argument("--ignore-certificate-errors")
options.add_argument("--window-size=1920,1080")
options.add_argument("--start-maximized")
webdriver.Chrome(
            settings.CHROME_WEBDRIVER,
            desired_capabilities=options.to_capabilities(),
            keep_alive=False,
            seleniumwire_options={
                'proxy': proxy,
                "disable_encoding": True
            }
        )

Hi @wkeeling I have added the configuration I used. Also it is working fine in 1.0.4.

wkeeling commented 5 years ago

@appunni-dishq Thanks. Have you configured the proxy server with authentication (I'm not able to tell from your example above because you're using variables settings.ANALYTICA_HTTPS_PROXY).

appunni-m commented 5 years ago

no

On Thu, 16 May 2019 at 19:04, Will Keeling notifications@github.com wrote:

@appunni-dishq https://github.com/appunni-dishq Thanks. Have you configured the proxy server with authentication (I'm not able to tell from your example above because you're using variables settings.ANALYTICA_HTTPS_PROXY).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wkeeling/selenium-wire/issues/49?email_source=notifications&email_token=AHQS4556GBPY3ZC3NH3FDCTPVVPF7A5CNFSM4HNIYXYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVR2KNI#issuecomment-493069621, or mute the thread https://github.com/notifications/unsubscribe-auth/AHQS45YI2PKZS6SSH43I3QDPVVPF7ANCNFSM4HNIYXYA .

wkeeling commented 3 years ago

This is no longer an issue on the latest version of selenium wire.

adsk2050 commented 1 year ago

I am getting this error when importing seleniumwire: ` File [\lib\site-packages\wsproto__init.py:9](file:////lib/site-packages/wsproto/init__.py:9) 1 """ 2 wsproto 3 ~~~ 4 5 A WebSocket implementation. 6 """ 7 from typing import Generator, Optional, Union ----> 9 from .connection import Connection, ConnectionState, ConnectionType 10 from .events import Event 11 from .handshake import H11Handshake

File [\lib\site-packages\wsproto\connection.py:23](file:////lib/site-packages/wsproto/connection.py:23) 21 from .extensions import Extension 22 from .frame_protocol import CloseReason, FrameProtocol, Opcode, ParseFailed ---> 23 from .utilities import LocalProtocolError 26 class ConnectionState(Enum): 27 """ 28 RFC 6455, Section 4 - Opening Handshake 29 """

File [\lib\site-packages\wsproto\utilities.py:12](file:////lib/site-packages/wsproto/utilities.py:12) 9 import os 10 from typing import Dict, List, Optional, Union ---> 12 from h11._headers import Headers as H11Headers 14 from .events import Event 15 from .typing import Headers

ImportError: cannot import name 'Headers' from 'h11._headers' (\lib\site-packages\h11_headers.py) `