SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
29.79k stars 8.03k forks source link

[🐛 Bug]: Enabling Downloads for Firefox requires geckodriver options #13379

Open titusfortner opened 6 months ago

titusfortner commented 6 months ago

What happened?

When server is started with: "--enable-managed-downloads true", you cannot start a firefox session.

It works when the grid sees:

session.capabilities={"acceptInsecureCerts": true,"browserName": "firefox","moz:debuggerAddress": true,"moz:firefoxOptions": {"log": {"level": "debug"}},"pageLoadStrategy": "normal","platformName": "mac","se:downloadsEnabled": true}

So it works so long as moz:firefoxOptions is not empty. I've seen this issue before and thought I fixed it. As I recall, the grid is creating a new capabilities object when downloads are enabled, but when firefoxOptions is empty it isn't using it, and then improperly sends the unrecognized capability to geckodriver which then fails (though it should error that it doesn't recognize that capability, so I'm not sure what is going on there).

How can we reproduce the issue?

options = webdriver.FirefoxOptions()
driver = webdriver.Remote(command_executor=server, options=options)

Relevant log output

09:40:24.106 DEBUG [SeleniumSpanExporter$1.lambda$export$4] - SpanData{spanContext=ImmutableSpanContext{traceId=21932467be9e5c97339cf775bd3cb044, spanId=32ebc9bed7b51060, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=21932467be9e5c97339cf775bd3cb044, spanId=541e9c7652c81dc0, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=true}, resource=Resource{schemaUrl=https://opentelemetry.io/schemas/1.20.0, attributes={service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.28.0"}}, instrumentationScopeInfo=InstrumentationScopeInfo{name=default, version=null, schemaUrl=null, attributes={}}, name=driver_service_factory.apply, kind=INTERNAL, startEpochNanos=1704037223125109224, endEpochNanos=1704037224105660995, attributes=AttributesMap{data={session.capabilities={"acceptInsecureCerts": true,"browserName": "firefox","moz:debuggerAddress": true,"pageLoadStrategy": "normal","platformName": "mac","se:downloadsEnabled": true}
, error=true}, capacity=128, totalAddedValues=2}, totalAttributeCount=2, events=[ImmutableEventData{name=exception, attributes={driver.url="http://localhost:46641", exception.message="Error while creating session with the driver service. Stopping driver service: Could not start a new session. Response code 500. Message: Process unexpectedly closed with status signal
Host info: host: 'SL-1837', ip: '2600:1700:7880:6620:0:0:0:21%en0'
Build info: version: '4.16.1', revision: '9b4c83354e'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '13.4.1', java.version: '17.0.7'
Driver info: driver.version: unknown", exception.stacktrace="org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Response code 500. Message: Process unexpectedly closed with status signal
Host info: host: 'SL-1837', ip: '2600:1700:7880:6620:0:0:0:21%en0'
Build info: version: '4.16.1', revision: '9b4c83354e'
System info: os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '13.4.1', java.version: '17.0.7'
Driver info: driver.version: unknown
    at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:140)
    at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:96)
    at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:68)
    at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:164)
    at org.openqa.selenium.grid.node.config.DriverServiceSessionFactory.apply(DriverServiceSessionFactory.java:72)
    at org.openqa.selenium.grid.node.local.SessionSlot.apply(SessionSlot.java:147)
    at org.openqa.selenium.grid.node.local.LocalNode.newSession(LocalNode.java:464)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.startSession(LocalDistributor.java:645)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor.newSession(LocalDistributor.java:564)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.handleNewSessionRequest(LocalDistributor.java:824)
    at org.openqa.selenium.grid.distributor.local.LocalDistributor$NewSessionRunnable.lambda$run$1(LocalDistributor.java:784)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
", exception.type="org.openqa.selenium.SessionNotCreatedException", logger="org.openqa.selenium.grid.node.config.DriverServiceSessionFactory", session.capabilities="{"acceptInsecureCerts": true,"browserName": "firefox","moz:debuggerAddress": true,"pageLoadStrategy": "normal","platformName": "mac","se:downloadsEnabled": true}
"}, epochNanos=1704037224104169755, totalAttributeCount=6}], totalRecordedEvents=1, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=ERROR, description=Kind: CANCELLED Description:}, hasEnded=true}

Operating System

Mac

Selenium version

4.16.1

What are the browser(s) and version(s) where you see this issue?

Firefox 120

What are the browser driver(s) and version(s) where you see this issue?

geckodriver 0.33

Are you using Selenium Grid?

4.16.1

titusfortner commented 6 months ago

@krmahadevan do you have time to look at this? I'm in the middle of several other things and don't want to jump down this particular rabbit hole right now. Thanks.

0xC4N1 commented 1 week ago

Having the same issue with docker selenium. Tried to get some insight with --log-level "ALL" but it doesn't seem very helpful. Any suggestions how to get more insight?

$ docker run -d -e SE_NODE_ENABLE_MANAGED_DOWNLOADS=true -e SE_LOG_LEVEL=ALL selenium/standalone-firefox:4.22.0-20240621
$ curl localhost:4444/status
{
  "value": {
    "ready": true,
    "message": "Selenium Grid ready.",
    "nodes": [
      {
        "id": "9c78b7de-380b-4a97-ace4-429f3842108d",
        "uri": "http:\u002f\u002f172.17.0.5:4444",
        "maxSessions": 1,
        "osInfo": {
          "arch": "amd64",
          "name": "Linux",
          "version": "5.4.0-186-generic"
        },
        "heartbeatPeriod": 30000,
        "availability": "UP",
        "version": "4.22.0 (revision c5f3146703)",
        "slots": [
          {
            "id": {
              "hostId": "9c78b7de-380b-4a97-ace4-429f3842108d",
              "id": "c59cd67b-d13c-4745-ac2c-0b461be8e5c4"
            },
            "lastStarted": "1970-01-01T00:00:00Z",
            "session": null,
            "stereotype": {
              "browserName": "firefox",
              "browserVersion": "127.0",
              "moz:firefoxOptions": {
                "binary": "\u002fusr\u002fbin\u002ffirefox"
              },
              "platformName": "linux",
              "se:downloadsEnabled": true,
              "se:noVncPort": 7900,
              "se:vncEnabled": true
            }
          }
        ]
      }
    ]
  }
}
>>> driver = Remote("http://localhost:4444/wd/hub")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/selenium/webdriver/remote/webdriver.py", line 152, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/lib/python3/dist-packages/selenium/webdriver/remote/webdriver.py", line 249, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/lib/python3/dist-packages/selenium/webdriver/remote/webdriver.py", line 318, in execute
    self.error_handler.check_response(response)
  File "/usr/lib/python3/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: Could not start a new session. New session request timed out 
Host info: host: 'f6045c8ff6cb', ip: '172.17.0.5'
Build info: version: '4.22.0', revision: 'c5f3146703'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-186-generic', java.version: '17.0.11'
Driver info: driver.version: unknown
14:41:11.296 DEBUG [RequestConverter.channelRead0] - Start of http request: DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /wd/hub/session HTTP/1.1
Host: 127.0.0.1:4444
Accept-Encoding: identity
Content-Length: 84
Accept: application/json
Content-Type: application/json;charset=UTF-8
User-Agent: selenium/4.0.0a1 (python linux)
Connection: keep-alive
14:41:11.299 DEBUG [RequestConverter.channelRead0] - Incoming message: DefaultLastHttpContent(data: PooledSlicedByteBuf(ridx: 0, widx: 84, cap: 84/84, unwrapped: PooledUnsafeDirectByteBuf(ridx: 327, widx: 327, cap: 2048)), decoderResult: success)
14:41:11.300 DEBUG [RequestConverter.channelRead0] - End of http request: DefaultLastHttpContent(data: PooledSlicedByteBuf(ridx: 84, widx: 84, cap: 84/84, unwrapped: PooledUnsafeDirectByteBuf(ridx: 327, widx: 327, cap: 2048)), decoderResult: success)
14:41:11.323 DEBUG [SpanWrappedHttpHandler.execute] - Wrapping request. Before OpenTelemetryContext{tracer=io.opentelemetry.sdk.trace.SdkTracer@551bdc27, context={}, span id=0000000000000000, trace id=00000000000000000000000000000000} and after OpenTelemetryContext{tracer=io.opentelemetry.sdk.trace.SdkTracer@551bdc27, context={opentelemetry-trace-span-key=SdkSpan{traceId=3936a1296e9b4d320009deaffa66fdbb, spanId=1758b3fbed896d1b, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, name=session_queue, kind=INTERNAL, attributes=AttributesMap{data={random.key=d23419b9-223e-4cad-b4e9-603d59893273}, capacity=128, totalAddedValues=1}, status=ImmutableStatusData{statusCode=UNSET, description=}, totalRecordedEvents=0, totalRecordedLinks=0, startEpochNanos=1719585671319058518, endEpochNanos=0}}, span id=1758b3fbed896d1b, trace id=3936a1296e9b4d320009deaffa66fdbb}
14:41:11.326 DEBUG [HttpTracing.inject] - Injecting (POST) /session into OpenTelemetrySpan{traceId=3936a1296e9b4d320009deaffa66fdbb,spanId=1758b3fbed896d1b} at org.openqa.selenium.remote.tracing.SpanWrappedHttpHandler:85
14:42:55.808 DEBUG [LocalDistributor.lambda$asRunnableHealthCheck$9] - Running healthcheck for Node http://172.17.0.5:4444
14:42:55.809 DEBUG [LocalDistributor.updateNodeAvailability] - Health check result for http://172.17.0.5:4444 was UP
14:44:55.807 DEBUG [LocalDistributor.lambda$asRunnableHealthCheck$9] - Running healthcheck for Node http://172.17.0.5:4444
14:44:55.808 DEBUG [LocalDistributor.updateNodeAvailability] - Health check result for http://172.17.0.5:4444 was UP
14:46:11.351 DEBUG [SeleniumSpanExporter$1.lambda$export$4] - SpanData{spanContext=ImmutableSpanContext{traceId=3936a1296e9b4d320009deaffa66fdbb, spanId=8a8eb21376ffe535, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=3936a1296e9b4d320009deaffa66fdbb, spanId=1758b3fbed896d1b, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=true, valid=true}, resource=Resource{schemaUrl=null, attributes={service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.39.0"}}, instrumentationScopeInfo=InstrumentationScopeInfo{name=default, version=null, schemaUrl=null, attributes={}}, name=sessionqueue.add_to_queue, kind=INTERNAL, startEpochNanos=1719585671335607066, endEpochNanos=1719585971350612266, attributes={}, totalAttributeCount=0, events=[], totalRecordedEvents=0, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=UNSET, description=}, hasEnded=true}
14:46:11.353 DEBUG [SeleniumSpanExporter$1.lambda$export$4] - SpanData{spanContext=ImmutableSpanContext{traceId=3936a1296e9b4d320009deaffa66fdbb, spanId=1758b3fbed896d1b, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=true}, parentSpanContext=ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}, resource=Resource{schemaUrl=null, attributes={service.name="unknown_service:java", telemetry.sdk.language="java", telemetry.sdk.name="opentelemetry", telemetry.sdk.version="1.39.0"}}, instrumentationScopeInfo=InstrumentationScopeInfo{name=default, version=null, schemaUrl=null, attributes={}}, name=session_queue, kind=INTERNAL, startEpochNanos=1719585671319058518, endEpochNanos=1719585971353526540, attributes=AttributesMap{data={http.status_code=500, http.method=POST, random.key=d23419b9-223e-4cad-b4e9-603d59893273, span.kind=server, http.target=/session}, capacity=128, totalAddedValues=5}, totalAttributeCount=5, events=[ImmutableEventData{name=HTTP request execution complete, attributes={http.flavor=1, http.handler_class="org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue", http.host="127.0.0.1:4444", http.method="POST", http.request_content_length="84", http.scheme="HTTP", http.status_code=500, http.target="/session", http.user_agent="selenium/4.0.0a1 (python linux)"}, epochNanos=1719585971353143043, totalAttributeCount=9}], totalRecordedEvents=1, links=[], totalRecordedLinks=0, status=ImmutableStatusData{statusCode=ERROR, description=Kind: INTERNAL Description:}, hasEnded=true}
14:46:11.356 WARN [SeleniumSpanExporter$1.lambda$export$3] - {"traceId": "3936a1296e9b4d320009deaffa66fdbb","eventTime": 1719585971353143043,"eventName": "HTTP request execution complete","attributes": {"http.flavor": 1,"http.handler_class": "org.openqa.selenium.grid.sessionqueue.local.LocalNewSessionQueue","http.host": "127.0.0.1:4444","http.method": "POST","http.request_content_length": "84","http.scheme": "HTTP","http.status_code": 500,"http.target": "\u002fsession","http.user_agent": "selenium\u002f4.0.0a1 (python linux)"}}
github-actions[bot] commented 1 week ago

This issue is looking for contributors.

Please comment below or reach out to us through our IRC/Slack/Matrix channels if you are interested.