Document.documentElement.outerHTML got null

integrii / headlessChrome

A Go package for working with headless Chrome. Run interactive JavaScript commands on web pages with Go and Chrome.

MIT License

120 stars 5 forks source link

Document.documentElement.outerHTML got null #3

Closed han2015 closed 6 years ago

han2015 commented 6 years ago

my config: i used the chrome canary

headlessChrome.ChromePath = `/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary`

I got this error on Mac OS. it seems the browser didn't start up in headless.

case <-startupTime.C:
    debug("ERROR: Browser failed to start before browser startup time cutoff")
    chromeSession.ForceClose() // force cloe the session because it failed
    err = errors.New("Chrome console failed to init in the alotted time")
    return &chromeSession, err

but i remove headless from args, the chrome could open a window right now. "--headless"

han2015 commented 6 years ago

I have 'fixed' my issue, but i think it has small bug.

When parse big site which has too many DOM content(js&css e.g https://google.com, https://www.baidu.com), the command document.documentElement.outerHTML will get itself, weird. it works well when you parse some cleaner and lighter site.
at some times, it will output result twice.(e.g. https://www.shanbay.com/web/account/login)
```
for len(browser.Output) > 0 {
fmt.Println(<-browser.Output)
}
```

will continue to researching that. after all, compare to some other chrome tools, this is most lightly and easily, very practical tool.

integrii commented 6 years ago

Make sure that you empty that browser.Output channel fully and wait for chrome to write all the console message it is going to write between your commands. It may have "old" data in it's queue. I think you can do something like:

for len(browser.Output) > 0 {
    <- browser.Output
}