rust-headless-chrome / rust-headless-chrome

A high-level API to control headless Chrome or Chromium over the DevTools Protocol. It is the Rust equivalent of Puppeteer, a Node library maintained by the Chrome DevTools team.
MIT License
2.16k stars 220 forks source link

DebugPortInUse error when running inside gVisor #261

Open nikitavbv opened 3 years ago

nikitavbv commented 3 years ago

Hi! Thank you for maintaining this crate!

It seems that browser process creation fails when running inside gVisor (used in environments like Google Cloud Run). Browser::new returns DebugPortInUse even though the port specified in LaunchOptions is actually free.

Steps to reproduce

Cargo.toml:

[package]
name = "gvisor-bug-reproduce"
version = "0.1.0"
authors = ["Nikita Volobuev <nikitavbv@gmail.com>"]
edition = "2018"

[dependencies]
headless_chrome={git="https://github.com/atroche/rust-headless-chrome"}
env_logger="0.8.4"
log = "0.4.14"

Here is src/main.rs, where we start a headless browser with a specific debugging port set (if we don't set it, it will give up after certain number of retries saying that no port was found):

use log::info;

use headless_chrome::{Browser, LaunchOptionsBuilder};
use headless_chrome::protocol::page::ScreenshotFormat;

fn main() {
    env_logger::init();

    info!("starting headless chrome...");

    let browser = Browser::new(LaunchOptionsBuilder::default()
        .headless(true)
        .sandbox(false)
        .port(Some(10531))
        .window_size(Some((1280, 720)))
        .build().unwrap()
    ).unwrap();
    let tab = browser.wait_for_initial_tab().unwrap();
    tab.navigate_to("https://google.com").unwrap();

    tab.capture_screenshot(ScreenshotFormat::PNG, None, true).unwrap();

    info!("done");
}

Dockerfile:

FROM ubuntu:21.10
WORKDIR /app

ARG DEBIAN_FRONTEND=noninteractive

# Install manually all the missing libraries
RUN apt-get update
RUN apt-get install -y gconf-service libasound2 libatk1.0-0 libcairo2 libcups2 libfontconfig1 libgdk-pixbuf2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libxss1 fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget

# Install Chrome
RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
RUN dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install

COPY target/release/gvisor-bug-reproduce /app/app

ENTRYPOINT [ "/app/app" ]

Build it using:

cargo build --release
docker build -t gvisor-bug-reproduce .

Now run the container:

docker run --env="RUST_LOG=info" gvisor-bug-reproduce

You may see the following output:

[2021-07-03T13:30:02Z INFO  gvisor_bug_reproduce] starting headless chrome...
[2021-07-03T13:30:02Z INFO  headless_chrome::browser::process] Launching Chrome binary at "/usr/bin/google-chrome-stable"
[2021-07-03T13:30:02Z INFO  headless_chrome::browser::process] Started Chrome. PID: 8
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::tab] Navigating a tab to https://google.com
[2021-07-03T13:30:03Z INFO  gvisor_bug_reproduce] done
[2021-07-03T13:30:03Z INFO  headless_chrome::browser] Dropping browser
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::process] Killing Chrome. PID: 8
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::transport::web_socket_connection] Sending shutdown message to message handling loop
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::transport] Received shutdown message
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::transport] Shutting down message handling loop
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::transport] cleared listeners, I think
[2021-07-03T13:30:03Z INFO  headless_chrome::browser] Finished browser's event handling loop
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::tab] finished tab's event handling loop
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::transport] dropping transport
[2021-07-03T13:30:03Z INFO  headless_chrome::browser::transport::web_socket_connection] dropping websocket connection

That is something you may expect to see - the browser was successfully started, the webpage was loaded and the screenshot was made.

Now let's try to run the same container inside gVisor. You can find this runtime in Cloud Run (inside Google Cloud), or by installing it locally and running using:

docker run --runtime=runsc --env="RUST_LOG=info" gvisor-bug-reproduce

You will probably get the following output:

[2021-07-03T14:53:51Z INFO  gvisor_bug_reproduce] starting headless chrome...
[2021-07-03T14:53:51Z INFO  headless_chrome::browser::process] Launching Chrome binary at "/usr/bin/google-chrome-stable"
[2021-07-03T14:53:51Z INFO  headless_chrome::browser::process] Started Chrome. PID: 2
[2021-07-03T14:53:52Z INFO  headless_chrome::browser::process] Killing Chrome. PID: 2
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: DebugPortInUse', src/main.rs:17:7
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Cause of this issue

If you take a look at chromium binary output, you may notice the following message:

[2021-07-03T14:55:06Z TRACE headless_chrome::browser::process] Chrome output: [0703/145506.975691:ERROR:address_tracker_linux.cc(214)] Could not bind NETLINK socket: Permission denied (13)

I am not sure completely at the moment what this error message means, but I do know that it can be ignored. If you don't interrupt the process the moment you see this message, the browser finishes the startup and webtools websocket becomes available at the port, which was specified in the LaunchOptions.

But the process actually gets interrupted. The regular expression in process.rs matches the error message above.

Proposed fix

One simple way to fix this problem would be to change this regexp to a more specific one or to ignore the Could not bind NETLINK socket message. I will send a PR soon with this change. Please let me know if you have any ideas for a better fix.

Thank you!

nikitavbv commented 3 years ago

Please see https://github.com/atroche/rust-headless-chrome/pull/262 for a fix.

mdrokz commented 3 years ago

Please see https://github.com/atroche/rust-headless-chrome/pull/262 for a fix.

Hello I will review this on Monday, i don't have access to my pc currently. Sorry for the delay