Headless mode improvements

[delphius]

In 2.4.0 there is a great new feature 'set_hide', wich let us use webui for background site testing.

We use webui in headless mode to test pages from other languages, while in order to load the pages themselves, we have to use tricks such as this one:

<script>
function loadData() {
  fetch("linktosomehost.com")
    .then(response => response.text())
    .then(content => {
      document.open("text/html");
      document.write(content);
      document.close();

      const scriptElement = document.createElement("script");
      scriptElement.src = "webui.js";
      document.body.appendChild(scriptElement);
    })
    .catch(error => {
      console.error(error);
    });
}
window.onload = loadData;
</script>

to let webui to load the necessary library code on the browser side to serve events and websocket connection it can be done by dom manipulating.

I certainly understand the true purpose of the library, but since the possibility of working in headless mode has already been implemented, maybe we should think about a more efficient way of working in this mode?

I would like to hear opinions and ideas in this direction.

And this is actual, since webui.js auto inject is removed

[hassandraga]

fetch() can be blocked by CORS. And bypass it using proxy needs authentication. Using --disable-web-security solves it but it shows a warning message on startup. I suggest using JavaScript like you did for those specific scenarios.

[fibodevy]

WebUI has control only over the content provided by itself (by CivetWeb http/websocket server). If you navigate to other URL, you lose all control over the browser.

You can kind of control DOM of other websites with tricks like your your JS, on some sites it will work, but some sites will defend (will you bypass Cloudflare check?). Your URL will always point to localhost, your port will be always different than 80 or 443.

CORS

You can modify your JS to not fetch('http://website') but fetch('/internal'), set up file handler, and on request to /internal download the site you want and return it. CORS will be bypassed.

You can disable CORS using this quick hack if you dont want to modify the lib:

webui_set_profile(window, 'profile', 'profie" --disable-web-security --something "');

File handler

After you set file handler, you will need to download all requests and return them, like a proxy.

Other possibilities

I can see more ways it can be achieved, but these are all hacks and wont be supported by the lib.

eg

• start Chromium with a profile that has Tampermonkey installed and configured to include webui.js on every website
• maybe there is command line switch to auto load custom extension?

But how to inject the script if you dont know the URL? Its dynamic, on localhost, that you will need to figure out too.

To sum up

WebUI has control only over the content it provided. Its not a web driver.