How to add custom headers

[lhomme-sans-pseudo]

Hello, I'm trying to find how to add custom headers. It would be very kind if you could help me. I found a solution using the seleniumwire module, the problem with this way of doing things is that we lose the advantage of undetected-chromedriver, against anti-programming request

Thanks you

[PePinodemrs]

You can only with an extension chrome.

        {
          "name": "yo",
          "description": "yoyoy",
          "version": "15",
          "manifest_version": 3,
          "permissions": [
            "declarativeNetRequest",
            "declarativeNetRequestFeedback",
            "scripting",
            "activeTab"
          ],
          "host_permissions": [ "*://*/*" ],
          "declarative_net_request": {
            "rule_resources": [
              {
                "id": "ruleset_1",
                "enabled": true,
                "path": "rules.json"
              }
            ]
          },
          "minimum_chrome_version": "%s"
        }
        """```

        ```rules_js = """
            [
              {
                "id" : 1,
                "priority": 1,
                "action": {
                  "type": "modifyHeaders",
                  "requestHeaders": [
                    { "header": "sec-ch-ua", "operation": "remove"}
                  ]
                },
                "condition" : {
                  "resourceTypes" : ["script", "main_frame", "sub_frame", "stylesheet", "image", "font", "object", "xmlhttprequest", "ping", "csp_report", "media", "websocket", "other"],
                  "requestMethod" : ["connect", "delete", "get", "head", "options", "patch", "post", "put"]
                }
              }
            ]
            """```You can set or remove, if you set you need to add to the dict "value":"what you want"

[kaliiiiiiiiii]

you can use modheader_selenium

[sebdelsol]

or use selenium-wire that's already compatible with undetected-chromedrive

[kaliiiiiiiiii]

or use selenium-wire that's already compatible with undetected-chromedrive

notice that https won't work just compatible with http

[PePinodemrs]

or use selenium-wire that's already compatible with undetected-chromedrive

and up to date ? or just follow your undetected chromedriver versions ?

[sebdelsol]

notice that https won't work just compatible with http

I'm not sure I understand what your problem with selenium-wire is.

and up to date ? or just follow your undetected chromedriver versions ?

Use my fork only if you have issues with prefs and/or need the service_creationflags otherwise I would NOT recommend my fork because I won't be able to do any maintenance in the near future.

The author is merging stuff from my fork so it should be fixed in the near future.

[sebdelsol]

Fixed prefs & service_creationflags are implemented in the new 3.1.7 version :muscle: :rocket:! Please python -m pip install undetected-chromedriver -U

EDIT: and close the issue

[kaliiiiiiiiii]

Implement in uc? @sebdelsol @ultrafunkamsterdam There's a way using the cdp protocoll.

# resource: https://stackoverflow.com/questions/66227508/selenium-4-0-0-beta-1-how-add-event-listeners-in-cdp

class cdp_listener:
    from typing import Dict

    def __init__(self):
        self.listeners = {}
    async def async_helper(self):
        async with driver.bidi_connection() as connection:
            session, devtools = connection.session, connection.devtools

            for listener in self.listeners.items():
                my_listener = await listener[1]["listener"](connection=connection)

                async for event in my_listener:
                    try:
                        await session.execute(listener[1]["at_event"](event=event, connection=connection))
                    except Exception as e:
                        print(e)

    def trio_helper(self):
        import trio
        trio.run(self.async_helper)

    def start_threaded(self, listeners: Dict[str,Dict[callable, callable]] = {}):
        if listeners:
            self.listeners = listeners

        import threading
        thread = threading.Thread(target=self.trio_helper)
        thread.start()
        return thread

    def add_listeners(self, listeners: Dict[str,Dict[callable, callable]]):
        self.listeners = listeners

    def remove_listener(self, listener:str):
        del self.listeners[listener]

async def all_requests(connection):
    session, devtools = connection.session, connection.devtools
    pattern = map(devtools.fetch.RequestPattern.from_json,[{"urlPattern":"*"}])
    pattern = list(pattern)
    await session.execute(devtools.fetch.enable(patterns=pattern))

    return session.listen(devtools.fetch.RequestPaused)

async def all_images(connection):
    session, devtools = connection.session, connection.devtools
    pattern = map(devtools.fetch.RequestPattern.from_json,[{"resourceType":"Image"}])
    pattern = list(pattern)
    await session.execute(devtools.fetch.enable(patterns=pattern))

    return session.listen(devtools.fetch.RequestPaused)

def connection_refused(event, connection):
    print({"type":event.resource_type.to_json(),"frame_id": event.frame_id, "url": event.request.url})

    session, devtools = connection.session, connection.devtools
    return devtools.fetch.fail_request(request_id=event.request_id,error_reason=devtools.network.ErrorReason.CONNECTION_REFUSED)

def continue_request(event, connection):
    print({"type":event.resource_type.to_json(),"frame_id": event.frame_id, "url": event.request.url})
    session, devtools = connection.session, connection.devtools

    headers = event.request.headers.to_json()

    my_headers = {"sec-ch-ua-platform": "Android"}
    headers.update(my_headers)
    my_headers = []
    for item in headers.items():
        my_headers.append(devtools.fetch.HeaderEntry.from_json({"name": item[0], "value": item[1]}))

    return devtools.fetch.continue_request(request_id=event.request_id, headers=my_headers)

cdp_listener = cdp_listener()
thread = cdp_listener.start_threaded(listeners= {"continue":{"listener":all_requests,"at_event":continue_request}})

driver.get('https://modheader.com/headers?product=ModHeader')

[tomdav999]

Here is my very simple solution (using javascript fetch api) to add custom headers to GET requests. I purposely return response.text() rather than response.json() and convert to json in python to avoid javascript errors when the response is empty. You could easily tweak this for POST and other types of requests.

def GET(self, driver, url, headers):
    response = driver.execute_async_script("var callback = arguments[arguments.length - 1]; fetch('" + url + "', {method: 'GET', headers: " + str(headers) + "}).then((response) => response.text().then((text) => callback({'status': response.status, 'text': text})))")
    return json.loads(response['text'])

[kaliiiiiiiiii]

def GET(self, driver, url, headers):
    response = driver.execute_async_script("var callback = arguments[arguments.length - 1]; fetch('" + url + "', {method: 'GET', headers: " + str(headers) + "}).then((response) => response.text().then((text) => callback({'status': response.status, 'text': text})))")
    return json.loads(response['text'])

Does work for single requests, but not a dynamic website. Dynamic websites make requests whilst navigating on the website based on need.

Also, you could just use node-js fetch or python requests without Selenium then :)

[Fakesum]

you can also use cdp Network.setExtraHTTPHeaders

driver = uc.Chrome(enable_cdp_events=True)

driver.execute_cdp_cmd("Network.setExtraHTTPHeaders", {
    "headers": {
        "<yourheader>": "<value>",
                ...
    }
})

driver.get("<url>") # the headers should be automatically added to every request.