SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
30.49k stars 8.15k forks source link

[šŸ› Bug]: Unable to hide DevTools listening on ws #13095

Closed MohamedV0 closed 10 months ago

MohamedV0 commented 11 months ago

What happened?

I'm working on a Selenium script to automate browser tasks, and I've run into an issue regarding the DevTools message. When I use the --headless option, the following message keeps appearing in the console:

"DevTools listening on ws://127.0.0.1:51481/devtools/browser/505e7f66-fd69-494b-a733-0cce438dec53"

However, the message successfully disappears when I remove the --headless option.

How can we reproduce the issue?

from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ['enable-logging'])
options.add_argument("--log-level=3")
options.add_argument("start-maximized")
options.add_argument("--headless")
driver = webdriver.Chrome(options=options)
input("")

Relevant log output

DevTools listening on ws://127.0.0.1:51481/devtools/browser/505e7f66-fd69-494b-a733-0cce438dec53

Operating System

Windows 11

Selenium version

Python 3.11.4 selenium 4.15.1

What are the browser(s) and version(s) where you see this issue?

Chrome Version 119.0.6045.106 (Official Build) (64-bit)

What are the browser driver(s) and version(s) where you see this issue?

Chrome Version 119.0.6045.106 (Official Build) (64-bit)

Are you using Selenium Grid?

no

github-actions[bot] commented 11 months ago

@MohamedV0, thank you for creating this issue. We will troubleshoot it as soon as we can.


Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

titusfortner commented 11 months ago

--headless is going away. Try --headless=new

MohamedV0 commented 11 months ago

--headless is going away. Try --headless=new

I tried this but same problem

nvborisenko commented 11 months ago

I think it is about that output from driver appears in user's console.

titusfortner commented 11 months ago
  1. Theoretically same output should appear for headless=new & without headless
  2. By default the driver output should be getting sent to dev/null
  3. You should try to manage driver logging with the Service class instead of the options class ā€” https://www.selenium.dev/documentation/webdriver/browsers/chrome/#log-output
matheusdorigo commented 11 months ago

My code has the same problem today, but i dont put any change

matheusdorigo commented 11 months ago

options = webdriver.EdgeOptions() options.add_argument("--headless") options.add_experimental_option('excludeSwitches', ['enable-logging'])

Configura o driver do Edge

driver = webdriver.Edge(service=Service(EdgeChromiumDriverManager().install()),options=options)

titusfortner commented 11 months ago
  1. you no longer need to use EdgeChromiumDriverManager. If you make sure all drivers are deleted from your system, it will just work.
  2. You do not need to excludeSwitches on enable-logging. Whether logs happen in stderr or to a file, should be managed by Service. https://source.chromium.org/chromium/chromium/src/+/main:chrome/test/chromedriver/chrome_launcher.cc;l=147
options = webdriver.EdgeOptions()
options.add_argument("--headless=new")
service = webdriver.EdgeService(log_output="/path/to/edgedriver.log")
driver = webdriver.Edge(options=options, service=service)
matheusdorigo commented 11 months ago

Hi there,

@titusfortner your reply dont work for me, the log still show up all errors and console output.

Add the exceptions worked for me:

options.add_argument('log-level=3')

I see the message DevTools listening on ws://127.0.0.1 but the rest of the logs, dont show to me, its great.

TomKranenburg commented 11 months ago

Same problem for me. I've tried every single thing on earth and I cannot get rid of the:

DevTools listening on ws://127.0.0.1:54805/devtools/browser/bc6af4eb-3a7c-4385-a820-cc932617e613

Message. Driving me crazy.

Only appears in headless mode. I'm using the Edge driver.

titusfortner commented 11 months ago

I can't duplicate this.

I have a devtools.py file with your original code:

from selenium import webdriver

options = webdriver.EdgeOptions()
options.add_experimental_option("excludeSwitches", ['enable-logging'])
options.add_argument("--log-level=3")
options.add_argument("--headless")
options.add_argument("start-maximized")
driver = webdriver.Edge(options=options)

driver.quit()

I run: python ./devtools.py and I do not see anything in the console.

TomKranenburg commented 11 months ago

I can't duplicate this.

I have a devtools.py file with your original code:

from selenium import webdriver

options = webdriver.EdgeOptions()
options.add_experimental_option("excludeSwitches", ['enable-logging'])
options.add_argument("--log-level=3")
options.add_argument("--headless")
options.add_argument("start-maximized")
driver = webdriver.Edge(options=options)

driver.quit()

I run: python ./devtools.py and I do not see anything in the console.

Running this code gives me the following output:

ConEmu64_xl2AqJ6b2K

titusfortner commented 11 months ago

Oh... maybe a windows thing

TomKranenburg commented 11 months ago

Oh... maybe a windows thing

Honestly this is progress if we can figure this out. Sorry also as I didn't know you could get Edge running on anything else. This is news to me if you can get it under other operating systems.

I gave up and just tanked the awkward debug message. Shame I can't get rid of it as it makes my code a bit ugly.

titusfortner commented 11 months ago

Edge is based on Chromium, so it works anywhere Chromium works, though, it tends to act weird on Linux for some reason. /shrug.

TomKranenburg commented 11 months ago

Makes sense. With this said I have a feeling this part of selenium was developed on a Linux system.

titusfortner commented 11 months ago

I can't duplicate it on Windows, either. Try turning on logging and see where it is coming from at least? https://www.selenium.dev/documentation/webdriver/troubleshooting/logging/

TomKranenburg commented 11 months ago
Selenium Manager binary found at: C:\Python310\lib\site-packages\selenium\webdriver\common\windows\selenium-manager.exe
Executing process: C:\Python310\lib\site-packages\selenium\webdriver\common\windows\selenium-manager.exe --browser MicrosoftEdge --debug --output json
msedgedriver not found in PATH
MicrosoftEdge detected at C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe
Running command: wmic datafile where name='C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe' get Version /value
Output: "\r\r\n\r\r\nVersion=119.0.2151.44\r\r\n\r\r\n\r\r\n\r"
Detected browser: MicrosoftEdge 119.0.2151.44
Required driver: msedgedriver 119.0.2151.44
msedgedriver 119.0.2151.44 already in the cache
Driver path: C:\Users\Citadel\.cache\selenium\msedgedriver\win64\119.0.2151.44\msedgedriver.exe
Browser path: C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe
Using driver at: C:\Users\Citadel\.cache\selenium\msedgedriver\win64\119.0.2151.44\msedgedriver.exe
Started executable: `C:\Users\Citadel\.cache\selenium\msedgedriver\win64\119.0.2151.44\msedgedriver.exe` in a child process with pid: 9644
POST http://localhost:52179/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "MicrosoftEdge", "pageLoadStrategy": "normal", "browserVersion": null, "ms:edgeOptions": {"excludeSwitches": ["enable-logging"], "extensions": [], "binary": "C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe", "args": ["--log-level=3", "--headless", "start-maximized"]}}}}
Remote response: status=200 | data={"value":{"capabilities":{"acceptInsecureCerts":false,"browserName":"msedge","browserVersion":"119.0.2151.44","fedcm:accounts":true,"ms:edgeOptions":{"debuggerAddress":"localhost:52184"},"msedge":{"msedgedriverVersion":"119.0.2151.44 (44557a29097af522913330e5b41f74810bdfb1a2)","userDataDir":"C:\\Windows\\SystemTemp\\scoped_dir9644_416605957"},"networkConnectionEnabled":false,"pageLoadStrategy":"normal","platformName":"windows","proxy":{},"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify","webauthn:extension:credBlob":true,"webauthn:extension:largeBlob":true,"webauthn:extension:minPinLength":true,"webauthn:extension:prf":true,"webauthn:virtualAuthenticators":true},"sessionId":"938b6beb42a4e34d30ebf065357c9d01"}} | headers=HTTPHeaderDict({'Content-Length': '830', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request
DELETE http://localhost:52179/session/938b6beb42a4e34d30ebf065357c9d01 {}
Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Content-Length': '14', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request

There we go.

titusfortner commented 11 months ago

So that is somehow changing the logging so you don't get the message at all? šŸ˜„

TomKranenburg commented 11 months ago

This is my code. Did I miss something? Still getting the output:

from selenium import webdriver
import logging

logger = logging.getLogger('selenium')
logger.setLevel(logging.DEBUG)
handler = logging.FileHandler("log.txt")
logger.addHandler(handler)

options = webdriver.EdgeOptions()
options.add_experimental_option("excludeSwitches", ['enable-logging'])
options.add_argument("--log-level=3")
options.add_argument("--headless")
options.add_argument("start-maximized")
driver = webdriver.Edge(options=options)

driver.quit()
titusfortner commented 11 months ago

I'm saying, I don't see the output in the logs you posted. Can you send everything to console so we can see the commands in order?

logger = logging.getLogger('selenium')
logger.setLevel(logging.DEBUG)
TomKranenburg commented 11 months ago

Done:

Selenium Manager binary found at: C:\Python310\lib\site-packages\selenium\webdriver\common\windows\selenium-manager.exe
Executing process: C:\Python310\lib\site-packages\selenium\webdriver\common\windows\selenium-manager.exe --browser MicrosoftEdge --debug --output json
msedgedriver not found in PATH
MicrosoftEdge detected at C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe
Running command: wmic datafile where name='C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe' get Version /value
Output: "\r\r\n\r\r\nVersion=119.0.2151.44\r\r\n\r\r\n\r\r\n\r"
Detected browser: MicrosoftEdge 119.0.2151.44
Reading msedgedriver version from https://msedgedriver.azureedge.net/LATEST_RELEASE_119_WINDOWS
Required driver: msedgedriver 119.0.2151.44
msedgedriver 119.0.2151.44 already in the cache
Driver path: C:\Users\Citadel\.cache\selenium\msedgedriver\win64\119.0.2151.44\msedgedriver.exe
Browser path: C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe
Using driver at: C:\Users\Citadel\.cache\selenium\msedgedriver\win64\119.0.2151.44\msedgedriver.exe
Started executable: `C:\Users\Citadel\.cache\selenium\msedgedriver\win64\119.0.2151.44\msedgedriver.exe` in a child process with pid: 1220
POST http://localhost:56280/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "MicrosoftEdge", "pageLoadStrategy": "normal", "browserVersion": null, "ms:edgeOptions": {"excludeSwitches": ["enable-logging"], "extensions": [], "binary": "C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe", "args": ["--log-level=3", "--headless", "start-maximized"]}}}}

DevTools listening on ws://127.0.0.1:56288/devtools/browser/bdabbde5-776e-4c6c-8462-05788500aa16
Remote response: status=200 | data={"value":{"capabilities":{"acceptInsecureCerts":false,"browserName":"msedge","browserVersion":"119.0.2151.44","fedcm:accounts":true,"ms:edgeOptions":{"debuggerAddress":"localhost:56288"},"msedge":{"msedgedriverVersion":"119.0.2151.44 (44557a29097af522913330e5b41f74810bdfb1a2)","userDataDir":"C:\\Windows\\SystemTemp\\scoped_dir1220_1969864134"},"networkConnectionEnabled":false,"pageLoadStrategy":"normal","platformName":"windows","proxy":{},"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify","webauthn:extension:credBlob":true,"webauthn:extension:largeBlob":true,"webauthn:extension:minPinLength":true,"webauthn:extension:prf":true,"webauthn:virtualAuthenticators":true},"sessionId":"49d884209b03ff99e2dd68dda106d861"}} | headers=HTTPHeaderDict({'Content-Length': '831', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request
DELETE http://localhost:56280/session/49d884209b03ff99e2dd68dda106d861 {}
Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Content-Length': '14', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
titusfortner commented 11 months ago

hmm, would be nice if we logged the arguments we started the driver with. I can add that for a future release. So, makes sense it records it when driver starts. Driver & Browser versions match, which might have been an issue.

I'm really not sure.

If you set browser_version = "118" or "117" do you still get it? Curious if something changed in the driver.

TomKranenburg commented 11 months ago

Fixed for me:

Selenium Manager binary found at: C:\Python310\lib\site-packages\selenium\webdriver\common\windows\selenium-manager.exe
Executing process: C:\Python310\lib\site-packages\selenium\webdriver\common\windows\selenium-manager.exe --browser MicrosoftEdge --browser-version 117 --debug --output json
msedgedriver not found in PATH
MicrosoftEdge detected at C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe
Running command: wmic datafile where name='C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe' get Version /value
Output: "\r\r\n\r\r\nVersion=119.0.2151.44\r\r\n\r\r\n\r\r\n\r"
Detected browser: MicrosoftEdge 119.0.2151.44
Discovered MicrosoftEdge version (119) different to specified browser version (117)
Checking MicrosoftEdge releases on https://edgeupdates.microsoft.com/api/products/?view=enterprise
Required browser: MicrosoftEdge 117.0.2045.60
Downloading MicrosoftEdge 117.0.2045.60 from https://msedge.sf.dl.delivery.mp.microsoft.com/filestreamingservice/files/04953c72-9e0d-420a-950c-15e0e659a225/MicrosoftEdgeEnterpriseX64.msi
Installing MicrosoftEdgeEnterpriseX64.msi
MicrosoftEdge 117.0.2045.60 is available at C:\Program Files (x86)\Microsoft\Edge\Application\117.0.2045.60\msedge.exe
Reading msedgedriver version from https://msedgedriver.azureedge.net/LATEST_RELEASE_117_WINDOWS
Required driver: msedgedriver 117.0.2045.60
Downloading msedgedriver 117.0.2045.60 from https://msedgedriver.azureedge.net/117.0.2045.60/edgedriver_win64.zip
Driver path: C:\Users\Citadel\.cache\selenium\msedgedriver\win64\117.0.2045.60\msedgedriver.exe
Browser path: C:\Program Files (x86)\Microsoft\Edge\Application\117.0.2045.60\msedge.exe
Using driver at: C:\Users\Citadel\.cache\selenium\msedgedriver\win64\117.0.2045.60\msedgedriver.exe
Started executable: `C:\Users\Citadel\.cache\selenium\msedgedriver\win64\117.0.2045.60\msedgedriver.exe` in a child process with pid: 26052
POST http://localhost:61619/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "MicrosoftEdge", "pageLoadStrategy": "normal", "browserVersion": null, "ms:edgeOptions": {"excludeSwitches": ["enable-logging"], "extensions": [], "binary": "C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\117.0.2045.60\\msedge.exe", "args": ["--log-level=3", "--headless", "start-maximized"]}}}}
Remote response: status=200 | data={"value":{"capabilities":{"acceptInsecureCerts":false,"browserName":"msedge","browserVersion":"117.0.2045.60","fedcm:accounts":true,"ms:edgeOptions":{"debuggerAddress":"localhost:61815"},"msedge":{"msedgedriverVersion":"117.0.2045.60 (a31cef5dc4c1f66dc0bb957c3250ba527e99b945)","userDataDir":"C:\\Windows\\SystemTemp\\scoped_dir26052_679090411"},"networkConnectionEnabled":false,"pageLoadStrategy":"normal","platformName":"windows","proxy":{},"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify","webauthn:extension:credBlob":true,"webauthn:extension:largeBlob":true,"webauthn:extension:minPinLength":true,"webauthn:extension:prf":true,"webauthn:virtualAuthenticators":true},"sessionId":"155950ccf5ff262896e5a230ff91081b"}} | headers=HTTPHeaderDict({'Content-Length': '831', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request
DELETE http://localhost:61619/session/155950ccf5ff262896e5a230ff91081b {}
Remote response: status=200 | data={"value":null} | headers=HTTPHeaderDict({'Content-Length': '14', 'Content-Type': 'application/json; charset=utf-8', 'cache-control': 'no-cache'})
Finished Request

Output is now:

ConEmu64_DTseq0OkVs

Nice and tidy. To anyone reading I downgraded to driver version 117 with:

options.set_capability("browserVersion", "117")

Full code example is:

from selenium import webdriver

options = webdriver.EdgeOptions()
options.add_experimental_option("excludeSwitches", ['enable-logging'])
options.add_argument("--log-level=3")
options.add_argument("--headless")
options.set_capability("browserVersion", "117")
options.add_argument("start-maximized")

driver = webdriver.Edge(options=options)
driver.quit()

This will disable the debug output for a short term fix. Thanks @titusfortner. You need anything else from me do let me know.

titusfortner commented 11 months ago

@TomKranenburg whenever we get the next release of Python out, it would be great if you could run it with logging on, I want to see what arguments the driver is getting started with.

TomKranenburg commented 11 months ago

@titusfortner will do. If it makes a difference I actually updated selenium in an effort to fix this. It was an old install.

matheusdorigo commented 11 months ago

@TomKranenburg dont works to me. Thanks.

matheusdorigo commented 11 months ago

Still show the log output

bhawna0612 commented 11 months ago

I also started getting all the console logs in the tests running output window after updating the Chrome browser to 119.0.6045.124. This is very annoying. Would there be a fix in the upcoming release?

diemol commented 10 months ago

This all points to be a behavior change in Chrome and ChromeDriver. Selenium is sending the same values as always. Please check with the Chrome folks.

github-actions[bot] commented 10 months ago

Hi, @MohamedV0. This issue has been determined to require fixes in ChromeDriver.

You can see if the feature is passing in the Web Platform Tests.

If it is something new, please create an issue with the ChromeDriver team. Feel free to comment the issues that you raise back in this issue. Thank you.

github-actions[bot] commented 9 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.