seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
4.46k stars 909 forks source link

Fetch Requests Data When Open a Page #2774

Closed adarmawan117 closed 1 month ago

adarmawan117 commented 1 month ago

I want to fetch all requests similar https://www.dilatoit.com/2020/12/17/how-to-capture-http-requests-using-selenium.html

from seleniumwire import webdriver  # Import from seleniumwire  

# Create a new instance of the Firefox driver  
driver = webdriver.Firefox()  

# Go to the Google home page  
driver.get('https://www.google.com')  

# Access and print requests via the `requests` attribute  
for request in driver.requests:  
    if request.response:  
        print(  
            request.url,  
            request.response.status_code,  
            request.response.headers['Content-Type'])  

How can i do with seleniumbase.?

I try to use

        for request in sb.driver.requests:
            if request.response:
                print(f"{request.url}\n{request.response.status_code}\n{request.response.headers['Content-Type']}")

But it give me error AttributeError: 'Chrome' object has no attribute 'requests'

mdmintz commented 1 month ago

You need wire mode enabled. First install the extra dependencies:

pip install seleniumbase[selenium-wire]

Then you can do this:

from seleniumbase import SB

with SB(wire=True, browser="firefox") as sb:
    sb.driver.get("https://wikipedia.org")
    for request in sb.driver.requests:
        print(request.url)