mattsse / chromiumoxide

Chrome Devtools Protocol rust API
Apache License 2.0
712 stars 69 forks source link

Page.goto hangs on some websites while browser.new_page works #191

Open krant opened 7 months ago

krant commented 7 months ago

For some use cases browser.new_page is not applicable (WebView not supporting it at all, also latency requirements) so I had to use page.goto. Unfortunately it hangs on some websites while browser.new_page handles them correctly:

use futures::StreamExt;
use chromiumoxide::browser::{Browser, BrowserConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let (mut browser, mut handler) =
        Browser::launch(BrowserConfig::builder()
            .build()?).await?;
    let handle = tokio::spawn(async move {
        while let Some(h) = handler.next().await {
            h.unwrap();
        }
    });

    let url = "https://time.com";

    // This works:
    //  let page = browser.new_page(url).await.unwrap();
    // This hangs and eventually timeouts:
        let page = browser.new_page("about:blank").await.unwrap(); // imitate re-use of existing page for brevity
        page.goto(url).await?;

    let content = page.content().await.unwrap();
    println!("{content}");

    browser.close().await?;
    browser.wait().await?;
    handle.await?;

    Ok(())
}

P.S. I've tried putting goto into separate task and then wait for Page.loadEventFired, and it was kinda working, but only for the first request - when second one arrives, the existing page still processes something from the previous request, which leads to large delays or timeouts.