Help: How to verify request header has been successfully changed

[nesbyte]

Hi,

Wondering how to verify that request header has been changed successfully. Followed the documentation and I am attempting to verify if the header is changing using Firefox console and it is not (checking in the session that is started by selenium in the network tab and reading the get request headers). Is this the correct way of checking if the request header has been changed? If not, how is it done? I have tried multiple different selenium-wire versions, so I assume the problem is on my end.

OS: Windows 10 Python version: Python 3.8.6 Attached code:

from seleniumwire import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
import time

import logging
logging.basicConfig(level=logging.INFO)

FirefoxBinary = r"C:\Program Files\Mozilla Firefox\firefox.exe"
driver = webdriver.Firefox(firefox_binary=FirefoxBinary, executable_path=r"./GeckoDriver/geckodriver.exe")

def interceptor(request):
    request.headers['My-Header'] = 'A value'

driver.request_interceptor = interceptor
time.sleep(5)
driver.get(r"https://www.google.com/")

driver.close()

Thanks

[wkeeling]

Thanks for raising this. Selenium Wire modifies the requests as they flow from the browser to the target site. This means that you won't see the change in the Firefox console as the header has not been modified at this point.

There are a couple of ways to inspect whether the header has been changed. First, Selelnium Wire captures requests after they have been modified, so you can simply print the captured requests and inspect the headers after the driver.get() call completes:

driver.get(r"https://www.google.com/")
for request in driver.requests:
    print(request)
    for name, value in request.headers.items():
        print(f'\t{name}: {value}')

Prints:

https://www.google.com/
    Host: www.google.com
    User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:84.0) Gecko/20100101 Firefox/84.0
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
    Accept-Language: en-US,en;q=0.5
    Accept-Encoding: gzip, deflate, br
    Connection: keep-alive
    Upgrade-Insecure-Requests: 1
    My-Header: A value

Alternatively, you can navigate to https://httpbin.org/headers which will echo back the headers to the browser so you can see them. You'll need to omit the driver.close() call so that Firefox remains open at the end.

[nesbyte]

Thank you for clearing things up. Tested both methods and they work perfectly.

To make sure I understand this properly: The reason why Firefox console does not see the change is due to the proxy which "changes" the request header after the request itself is sent? So request is sent through Firefox (hence the console does not see the edit), proxy picks it up, changes it and then passes it on. Is that correct?

[wkeeling]

Yes that's exactly right.

Also, don't forget that if you're wanting to replace a header, you'll need to remove the existing header first:

def interceptor(request):
    del request.headers['Existing-Header']
    request.headers['Existing-Header'] = 'A value'

[nesbyte]

Great. Noticed that in the documentation as well. Thank you very much for your help. Really appreciate it.