search

mozilla / geckodriver

WebDriver for Firefox
https://firefox-source-docs.mozilla.org/testing/geckodriver/
Mozilla Public License 2.0
7.09k stars 1.52k forks source link

https://bet365.com prevents itself to be automated by freezing the browser #1787

Closed ghostwords closed 3 years ago

ghostwords commented 3 years ago

System

Testcase

Visiting a particular website (bet365.com) breaks the tab for subsequent navigation somehow. geckodriver log suggests that the website finished loading, but the tab is now in a strange state where navigating elsewhere hangs perpetually. No timeouts seem to apply.

Here is a demo script that should reproduce the problem:

#!/usr/bin/env python3

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

options = Options()
options.binary = "/usr/bin/firefox" # FIXME absolute path to your Firefox binary
options.log.level = "trace"

driver = webdriver.Firefox(options=options)

driver.get("https://bet365.com/")

# we never get here
driver.get("https://example.com/")

driver.quit()

Trace-level log

geckodriver.log ``` 1602188604613 geckodriver INFO Listening on 127.0.0.1:38700 1602188604620 mozrunner::runner INFO Running command: "/usr/bin/firefox" "--marionette" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofile2A1aOe" 1602188604620 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:40105 1602188605336 Marionette TRACE Marionette enabled 1602188605423 Marionette TRACE Received observer notification toplevel-window-ready console.error: SearchCache: "_readCacheFile: Error reading cache file:" (new Error("", "(unknown module)")) 1602188607450 Marionette TRACE Received observer notification marionette-startup-requested 1602188607451 Marionette TRACE Waiting until startup recorder finished recording startup scripts... 1602188607472 Marionette TRACE All scripts recorded. 1602188607473 Marionette DEBUG Setting recommended pref apz.content_response_timeout to 60000 1602188607473 Marionette DEBUG Setting recommended pref browser.contentblocking.introCount to 99 1602188607473 Marionette DEBUG Setting recommended pref browser.download.panel.shown to true 1602188607474 Marionette DEBUG Setting recommended pref browser.newtabpage.enabled to false 1602188607475 Marionette DEBUG Setting recommended pref browser.pagethumbnails.capturing_disabled to true 1602188607475 Marionette DEBUG Setting recommended pref browser.search.update to false 1602188607475 Marionette DEBUG Setting recommended pref browser.tabs.disableBackgroundZombification to false 1602188607475 Marionette DEBUG Setting recommended pref browser.tabs.remote.separatePrivilegedContentProcess to false 1602188607476 Marionette DEBUG Setting recommended pref browser.tabs.unloadOnLowMemory to false 1602188607476 Marionette DEBUG Setting recommended pref browser.tabs.warnOnCloseOtherTabs to false 1602188607476 Marionette DEBUG Setting recommended pref browser.tabs.warnOnOpen to false 1602188607476 Marionette DEBUG Setting recommended pref browser.usedOnWindows10.introURL to 1602188607476 Marionette DEBUG Setting recommended pref browser.urlbar.suggest.searches to false 1602188607477 Marionette DEBUG Setting recommended pref datareporting.policy.dataSubmissionPolicyAccepted to false 1602188607478 Marionette DEBUG Setting recommended pref dom.disable_beforeunload to true 1602188607478 Marionette DEBUG Setting recommended pref dom.disable_open_during_load to false 1602188607478 Marionette DEBUG Setting recommended pref dom.file.createInChild to true 1602188607478 Marionette DEBUG Setting recommended pref dom.max_chrome_script_run_time to 0 1602188607478 Marionette DEBUG Setting recommended pref dom.max_script_run_time to 0 1602188607479 Marionette DEBUG Setting recommended pref dom.push.connection.enabled to false 1602188607479 Marionette DEBUG Setting recommended pref extensions.getAddons.cache.enabled to false 1602188607479 Marionette DEBUG Setting recommended pref extensions.getAddons.discovery.api_url to data:, 1602188607479 Marionette DEBUG Setting recommended pref network.http.prompt-temp-redirect to false 1602188607479 Marionette DEBUG Setting recommended pref privacy.trackingprotection.enabled to false 1602188607480 Marionette DEBUG Setting recommended pref security.fileuri.strict_origin_policy to false 1602188607480 Marionette DEBUG Setting recommended pref security.notification_enable_delay to 0 1602188607480 Marionette DEBUG Setting recommended pref signon.autofillForms to false 1602188607482 Marionette DEBUG Setting recommended pref signon.rememberSignons to false 1602188607542 Marionette INFO Listening on port 40105 1602188607542 Marionette DEBUG Marionette is listening 1602188607627 Marionette DEBUG Accepted connection 0 from 127.0.0.1:59512 1602188607627 geckodriver::marionette DEBUG Connection to Marionette established on 127.0.0.1:40105. 1602188607628 Marionette DEBUG 0 -> [0,1,"WebDriver:NewSession",{"acceptInsecureCerts":true,"browserName":"firefox"}] 1602188607628 Marionette WARN TLS certificate errors will be ignored for this session 1602188607651 Marionette TRACE [15] Frame script loaded 1602188607652 Marionette TRACE [15] Frame script registered 1602188607658 Marionette DEBUG 0 <- [1,1,null,{"sessionId":"c289f9a1-05cf-4346-a723-1c3982642ff0","capabilities":{"browserName":"firefox","browserVersion":"81.0" ... /rust_mozprofile2A1aOe","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}] 1602188607662 webdriver::server DEBUG <- 200 OK {"value":{"sessionId":"c289f9a1-05cf-4346-a723-1c3982642ff0","capabilities":{"acceptInsecureCerts":true,"browserName":"firefox","browserVersion":"81.0","moz:accessibilityChecks":false,"moz:buildID":"20200917005511","moz:geckodriverVersion":"0.27.0","moz:headless":false,"moz:processID":9978,"moz:profile":"/tmp/rust_mozprofile2A1aOe","moz:shutdownTimeout":60000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true,"pageLoadStrategy":"normal","platformName":"linux","platformVersion":"4.4.0-177-generic","rotatable":false,"setWindowRect":true,"strictFileInteractability":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"unhandledPromptBehavior":"dismiss and notify"}}} 1602188607664 webdriver::server DEBUG -> POST /session/c289f9a1-05cf-4346-a723-1c3982642ff0/url {"url": "https://bet365.com/"} 1602188607665 Marionette DEBUG 0 -> [0,2,"WebDriver:Navigate",{"url":"https://bet365.com/"}] 1602188607677 Marionette TRACE [15] Received DOM event beforeunload for about:blank 1602188609387 Marionette TRACE [15] Received DOM event pagehide for about:blank 1602188609601 Marionette TRACE [15] Received DOM event DOMContentLoaded for https://www.bet365.com/ 1602188609607 Marionette TRACE [15] Received DOM event pageshow for https://www.bet365.com/ 1602188609608 Marionette DEBUG 0 <- [1,2,null,{"value":null}] 1602188609613 webdriver::server DEBUG <- 200 OK {"value":null} 1602188609615 webdriver::server DEBUG -> POST /session/c289f9a1-05cf-4346-a723-1c3982642ff0/url {"url": "https://example.com/"} 1602188609616 Marionette DEBUG 0 -> [0,3,"WebDriver:Navigate",{"url":"https://example.com/"}] 1602188609628 Marionette TRACE [15] Received DOM event beforeunload for https://www.bet365.com/ ```
ghostwords commented 3 years ago

Correction to my comment in the source code above: we do get to driver.get("https://example.com/"), that's the part that gets stuck. It's just that the browser doesn't ever seems to leave bet365.com.

ghostwords commented 3 years ago

There should be some sort of timeout that takes care of forcibly terminating whatever page scripts are preventing leaving the page (if that's what is happening here).

whimboo commented 3 years ago

Thanks for letting us know. After some investigation this actually seems to be a problem with Firefox. I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1670181 for that.

whimboo commented 3 years ago

Actually the page prevents itself from being automated in a way that is not nice. See the referenced bug for more details.

whimboo commented 3 years ago

Closing this issue given that it's Firefox and there is nothing we can do for Marionette nor geckodriver.