Page Never Completely Loads

I cloned and built beta 2.1 a few hours ago and I created a small script to try out the new BrowserMobProxy interface with WebDriver. I crammed everything into main() to make it simple to replicate.

I tested out a few large sites (Google, Yahoo, Weather, etc) and weather.com won't finish loading - the other sites complete in a few seconds.

public static void main(String[] args) throws InterruptedException {

    // final String URL = "http://google.com";
    // final String URL = "http://yahoo.com";
    // final String URL = "http://msn.com";
    final String URL = "http://weather.com";

    BrowserMobProxy server = new BrowserMobProxyServer();
    server.start();
    Proxy proxy = ClientUtil.createSeleniumProxy(server);

    DesiredCapabilities caps = new DesiredCapabilities();
    caps.setCapability(CapabilityType.PROXY, proxy);
    WebDriver driver = new ChromeDriver(caps);

    driver.get(URL);
    Thread.sleep(2000);
    System.out.println(server.getHar().getLog().getEntries().size());

    driver.quit();
    server.stop();
}

Here are my capabilities:

Capabilities [{
    applicationCacheEnabled=false,
    rotatable=false,
    mobileEmulationEnabled=false,
    chrome={userDataDir=C:\Users\rlevell\AppData\Local\Temp\scoped_dir10856_862},
    takesHeapSnapshot=true,
    databaseEnabled=false,
    handlesAlerts=true,
    version=42.0.2311.90,
    platform=XP,
    browserConnectionEnabled=false,
    nativeEvents=true,
    acceptSslCerts=true,
    locationContextEnabled=true,
    webStorageEnabled=true,
    browserName=chrome,
    takesScreenshot=true,
    javascriptEnabled=true,
    cssSelectorsEnabled=true}
]

The jars I am using:

    <dependency>
        <groupId>net.lightbody.bmp</groupId>
        <artifactId>browsermob-core</artifactId>
        <version>2.1.0-beta-1-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>net.lightbody.bmp</groupId>
        <artifactId>browsermob-core-littleproxy</artifactId>
        <version>2.1.0-beta-1-SNAPSHOT</version>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>2.45.0</version>
    </dependency>

I am using ChromeDriver version 2.15.322448. The same thing happened when using Firefox, but I could not rely on it as my Firefox version is 37 and I don't believe FirefoxDriver supports it yet. The page loads fine when I comment out the BrowserMob code.

Hi jekh,

I'm trying to load webpages, wait until they've finished loading, scroll to their bottoms, and wait for until they've finished loading. Because the load times are indeterminate, I am using browsermob-proxy to wait for traffic to stop before scrolling. I am using Python via Anaconda and Jupyter-Notebook. The following code consistently prints below my Jupyter-Notebook cell a duration of about 19,900 milliseconds, which is the difference between the wait time limit and the length of the quiet period, which I think is basically the amount of time wait_for_traffic_to_stop is allowed to wait.

# Record the present time.
start_time = time.time()

# Have the proxy server have Chrome load Google.
driver.get("https://www.google.com/")

# Wait for traffic to stop.
status_code = proxy.wait_for_traffic_to_stop(100, 20000)

# Print the time between load beginning and traffic end, or wait time limit.
duration = time.time() - start_time
print(duration)

That all being said, in Anaconda Prompt, I executed the command "pip install git+https://github.com/AutomatedTester/browsermob-proxy-py.git". Despite browsermob-proxy-py's documentation seeming to imply that my command would install browsermob-proxy 2.0, my command installed browsermob-proxy 0.8.0. Given a previous post of yours in this thread, it seems that the above time-out issue was fixed in browsermob-proxy 2. How do I install browsermob-proxy 2 or later via Anaconda Prompt?

lightbody / browsermob-proxy

Page Never Completely Loads #212