search

rstudio / chromote

Chrome Remote Interface for R
https://rstudio.github.io/chromote/
156 stars 20 forks source link

Headless Chrome updates #171

Open gadenbuie opened 2 months ago

gadenbuie commented 2 months ago

I'm opening this issue mostly to stash some things I've learned while debugging #170.

Chrome's headless mode was previously an entirely separate web browser from headful Chrome. Last year, Chrome introduced a new headless mode that uses the same browser implementation with the UI turned off, introduced as --headless=new. Old headless mode was previously available as --headless=old or --headless.

Chrome is now in the process of transitioning the default value of --headless to --headless=new.

Note: Passing the --headless command-line flag without an explicit value still activates the old Headless mode, but we intend to change this default. We intend to remove the old Headless from the Chrome binary and stop supporting this mode in Puppeteer in 2024. Simultaneously, we'll make old Headless mode available as a standalone binary for users who can't upgrade yet.

I haven't found an official statement saying that this change happened, but clearly --headless is now --headless=new. At some point in the future, --headless=old will be removed and users will be expected to use a separate chrome-headless-shell binary, which is already available.

launch_chrome(), by way of launch_chrome_impl(), forces --headless

https://github.com/rstudio/chromote/blob/f0302851f84960fabb2e14509e42b7a46a193877/R/chrome.R#L175-L180

but we could ease the transition by allowing users to set this value themselves. We could:

  1. Move --headless into default_chrome_args(), but this makes it easy to accidentally forget to include the --headless flag. OTOH, that might be desirable given that it is possible to use the Chrome DevTools Protocol to drive headful Chrome.

  2. Add --headless when launching Chrome only if --headless or --headless=* are not included in args.

We should also test that chromote is compatible with chrome-headless-shell.

gadenbuie commented 2 months ago

Download chrome-headless-shell from https://googlechromelabs.github.io/chrome-for-testing/#stable

The current easiest way to get chrome-headless-shell is to use pupeteer and npx. Store the downloaded binary somewhere stable using --path, like ~/.browsers:

npx @puppeteer/browsers install chrome-headless-shell --path ~/.browsers
Downloading chrome-headless-shell 130.0.6684.0 - 86.8 MB [====================] 100% 0.0s 
chrome-headless-shell@130.0.6684.0 /Users/garrick/.browsers/chrome-headless-shell/mac_arm-130.0.6684.0/chrome-headless-shell-mac-arm64/chrome-headless-shell

This outputs the path to the binary, which you can then use in the CHROMOTE_CHROME envvar:

Sys.setenv(CHROMOTE_CHROME = "/Users/garrick/.browsers/chrome-headless-shell/mac_arm-130.0.6684.0/chrome-headless-shell-mac-arm64/chrome-headless-shell")

At that point, chromote will use the chrome-headles-shell, i.e. --headless=old at a stable version.

hadley commented 2 months ago

Is there some reason that chromote can't use the new headless mode? What am I missing?

gadenbuie commented 2 months ago

chromote can use the new headless mode. I think Chrome intended for the new headless mode to be a drop-in replacement for the old headless mode but that hasn't been the case. There are many instances of differences and broken behavior between the two modes, the biggest of which is that you cannot simultaneously open headless and headfull Chrome instances on Windows.

We'll benefit both from giving Chrome some time to work out these issues and from any testing with the new headless mode that we or chromote users can do in the mean time.