go-rod / rod

A Chrome DevTools Protocol driver for web automation and scraping.
https://go-rod.github.io
MIT License
5.27k stars 346 forks source link

Q: Freezing when MustPage(url) with headless chromium #399

Closed nea closed 2 years ago

nea commented 3 years ago

Rod Version: v0.97.11

What have you tried to solve the question

Hi

I have a headless chromium started, running and go-rod connect to it. The connection looks good, starts normally without any issues.

If I then create a browser.MustConnect() also all goes well until I try to open MustPage(url). No error, no nothing, it just hangs. I tried to Page(url) if an error is returned but nothing.

The headless shell is running with disable-gpu, single-process, no-sandbox on an aws lambda instance. If I run a dump-dom on the headless shell it runs and prints without issues, but connecting and trying to navigate freezes the call. I also tried to enable logging but havent gotten anything out of it.

Anybody got an idea what I am missing here?

Thanks

ysmood commented 3 years ago

Sorry, we can't reproduce it. Can you provide minimal code and steps for us to reproduce your issue?

deehidayat commented 3 years ago

Hi, I got same issue on Windows 7. If I run on headless mode it do nothing when MustPage(url) (freezed). I tried to open monitoring server it seems page do nothing, tab title still . If I run headfull mode all went well except if I minimize browser, it like freeze again. Have try "Run as administrator" but still.

    log.Println("<<< Run Launcher >>>")
    l := launcher.New().
        Headless(appConfig.Headless).
        Devtools(false)

    defer l.Cleanup() // remove user-data-dir

    url := l.MustLaunch()

    path, _ := launcher.LookPath()
    log.Printf("Launcher path %s", path)

    log.Println("<<< Run Browser >>>")
    // Trace shows verbose debug information for each action executed
    // Slowmotion is a debug related function that waits 2 seconds between
    // each action, making it easier to inspect what your code is doing.
    browser := rod.New().
        ControlURL(url).
        Trace(appConfig.Debug).
        SlowMotion(appConfig.Delay).
        MustConnect()

    if appConfig.Headless {
        urlMonitor := browser.ServeMonitor("")
        log.Printf("Monitoring URL: %s", urlMonitor)
    }

    defer browser.MustClose()

    log.Printf("<<< Goto Login Page >>>")
    page := browser.MustPage(LOGIN_URL).MustWaitLoad()  // Program Stuck Here 

Note: same program works well on Windows 10

ysmood commented 3 years ago

@deehidayat will any website freeze?

nea commented 3 years ago

Ahoi @deehidayat

I don't know if this fixes your case too, but for me the issue was using --disable-gpu in my environment. Removing and providing swiftshader as gl fallback worked flawlessly. In my case it was no Windows but a Linux environment, but maybe worth a try.

Best

ysmood commented 3 years ago

@nea How about we add --disable-gpu as the default? It will be great if you could make a PR for it, just add it after this line:

https://github.com/go-rod/rod/blob/3b631c5a5734c47b02a57fbb417990cb195ad097/lib/launcher/launcher.go#L74

nea commented 3 years ago

Ahoi @ysmood

In general I would agree to add --disable-gpu as default because in most cases this probably is the preferred way. But for me it was the opposite: I had to remove --disable-gpu and replace it with --use-gl=swiftshader in my case.

So maybe this should rather be a documented suggestion than a default?

Cheers

ysmood commented 3 years ago

@nea So you mean if you don't use swiftshader, you still have problems? I thought at least it won't freeze anymore.

nea commented 3 years ago

Ahoi @ysmood

Exactly. I retried it several times, recompiled and tested and it was reproducible:

Environment was a headless Linux container (serverless), why it might be a very specific case in my test, but at least these were my findings.

ysmood commented 3 years ago

@nea how about make swiftshader the default? Have you tried this docker image? I never use swiftshader, it works fine for me.

nea commented 3 years ago

Ahoi @ysmood

Didn't even know about that Docker image ^^' As I used it in a Lambda environment, I referred back to the AWS images and did not compile my own. If this image works, then this could be a solution for serverless environments, too, as most providers allow container-based deployments nowadays. Damn, should have known earlier ^^ Swiftshader would basically just be a "software-renderer" so that no HW limitation should fail the local system. It is the absolute fallback, so to say ^^

ysmood commented 3 years ago

@nea about how to use this image https://github.com/go-rod/rod/blob/master/lib/examples/connect-browser/main.go

ysmood commented 3 years ago

@nea also check this one https://go-rod.github.io/#/custom-launch?id=remotely-manage-the-launcher

egonelbre commented 3 years ago

I had a similar issue with browser hanging, one thing that helped me to debug the issue was to add a logger for the launcher.

For example:

l := launcher.New().
    Logger(os.Stderr)
...

Then the browser startup errors will be printed to stderr.