mafredri / cdp

Package cdp provides type-safe bindings for the Chrome DevTools Protocol (CDP), written in the Go programming language.
MIT License
725 stars 45 forks source link

LoadEventFired truble #98

Closed biter777 closed 5 years ago

biter777 commented 5 years ago

Hi!

Need some help, if possible. I have a navigate func, but loadEventFired.Recv() always catches a "deadline". If i use a "t.Page.DOMContentEventFired(ctx)" instead of "t.Page.LoadEventFired(ctx)" its ok. I tried both Chrome and Chromium.

What can be wrong?

func navigate(ctx context.Context, url string) error {
    // Make sure Page events are enabled.
    err := t.Page.Enable(ctx)
    if err != nil {
        return err
    }

    loadEventFired, err := t.Page.LoadEventFired(ctx)
    if err != nil {
        return err
    }
    defer loadEventFired.Close()

    nav, err := t.Page.Navigate(ctx, page.NewNavigateArgs(url)) 
    if err != nil {
        return err
    }
    if nav.ErrorText != nil {
        return fmt.Errorf("navigation failed: %v", *nav.ErrorText)
    }

    _, err = loadEventFired.Recv() // ALWAYS ERR: <context.deadlineExceededError>
    return err
}
mafredri commented 5 years ago

Hi, it looks like the page isn't loading for some reason. I would recommend enabling logging for the protocol, it can give you some information on why it isn't working.

Example here.

biter777 commented 5 years ago

Thanks for your response!

I did some tests. Event "load" does not come if the site uses some js-scripts. I do not know why this is happening.

I'm currently using SetLifecycleEventsEnabled, and waiting for the event "networkIdle". It works fine, but not kosher. )

Tell me, if possible, how can I get a list of frames on the page and organize the waiting load for each of the frames? I tried through Debugger interface, but without success.

mafredri commented 5 years ago

There was a similar issue previously in #67, probably a bug in Chrome. As an alternative you could try DOMContentEventFired instead. Still though, waiting for networkIdle could be a pretty good solution since there's no guarantee that resources have been loaded by the time LoadEventFired or DOMContentEventFired happen.

biter777 commented 5 years ago

Still, as an option, we can disable unnecessary scripts on the page via Emulation.SetScriptExecutionDisabled(). But I do not know how to get a list of scripts on the page.

AlaNDGT commented 5 years ago

Hello,

Was exactly looking for how we could listen for the networkidle event.

How can we achieve that with the package?

Thanks in advance!

mafredri commented 5 years ago

@AlaNDGT use the Page domain to first enable lifecycle events and then listen to them via the event client.

biter777 commented 5 years ago

Hello,

Was exactly looking for how we could listen for the networkidle event.

How can we achieve that with the package?

Thanks in advance!

err = t.Page.SetLifecycleEventsEnabled(ctx, page.NewSetLifecycleEventsEnabledArgs(true))
...
events, err := t.Page.LifecycleEvent(ctx)
...
defer events.Close()

for ... {
    event, err := events.Recv()
    ...
}
AlaNDGT commented 5 years ago

@mafredri @biter777 Oh thanks!