wkeeling / selenium-wire

Extends Selenium's Python bindings to give you the ability to inspect requests made by the browser.
MIT License
1.9k stars 254 forks source link

Can we have the webdriver run forever? #283

Closed dolfindev closed 3 years ago

dolfindev commented 3 years ago

Is it possible to have the webdriver run forever and save the ws message into a file? Basically I would like to capture the data stream from websocket using selenium wire.

wkeeling commented 3 years ago

Sorry for the delay in responding.

Yes you can do this - see the example code below. Once the browser has opened up the websocket echo page, click the "Connect" button and try sending some websocket messages. You'll see them printed to the terminal. You can change the print(message) line to write the messages to a file.

import time
driver = webdriver.Chrome()
driver.get('https://www.websocket.org/echo.html')

request = driver.wait_for_request('wss://echo.websocket.org', timeout=30)

while True:
    try:
        message = request.ws_messages.pop(0)
    except IndexError:
        request = driver.wait_for_request('wss://echo.websocket.org')
        time.sleep(0.2)
    else:
        print(message)  # Change this if you want to write to a file instead
dolfindev commented 3 years ago

@wkeeling Thank you very much.

dolfindev commented 3 years ago

Actually what I want to ask is different.

If I do the following way, I can print all the message from websocket and I do not need to establish the connection to websocket. Is it possible to keep the following code run forever and print the message from websocket on the fly.

driver.get('https://www.websocket.org/echo.html')
time.sleep(10)
for request in driver.requests:
    if request.ws_messages:
        for j in request.ws_messages:
            print(
                j.content,
                j.date,
                j.from_client
            )
dolfindev commented 3 years ago

The following code works. I think it is because the message 'wss://echo.websocket.org' is received multiple time before a good connection to the websocket is established. Using request = driver.wait_for_request('wss://echo.websocket.org', timeout=30) catches only the first message.

    driver.get('https://www.websocket.org/echo.html')
    time.sleep(10)
    for request in driver.requests:
        if request.ws_messages:
            while True:
                try:
                    message = request.ws_messages.pop(0)
                except IndexError:
                    time.sleep(1.0)
                else:
                    print(
                        message.content,
                        message.date,
                        message.from_client
                    )