Manually version browsers

@io_bazel_rules_webtesting doesn't appear to be maintained anymore and hasn't see a release in over 2 years. This means the version of downloaded browsers is falling out of date.

It wouldn't be that big a deal, except that I've been using <template shadowroot="open"> because that is what's supported in the version of Chrome used for testing. However, the spec was changed to use <template shadowrootmode="open"> and Chrome M119 is slated to remove support for <template shadowroot="open">. This is currently scheduled for Nov 7th, per https://chromiumdash.appspot.com/schedule.

As a result, I need to update my code to use shadowrootmode, which means I need a recent-enough browser version which supports that. I'm wondering if I can fork or build a simpler solution good enough for @rules_prerender. This is definitely blocking 1.0.0 because I wouldn't want to ship that until I can use shadowrootmode correctly.

I have a prototype in ref/browser-update forked from an implementation in the Angular Dev Infra repository. This was sufficient to download recent versions of Chrome / Firefox and get a passing test with shadowrootmode on Chrome.

Some more things I'll need to figure out:

[ ] Firefox - Doesn't seem to be starting correctly in WSL right now. I would love to do better cross-browser testing, but Firefox doesn't support declarative shadow DOM right now, which is a big part of @rules_prerender. It might be fine since I have the polyfill, and could serve as a test case for that. But we may want to think about how much value this is actually giving and whether or not its worth it.
[X] Debugging - I'm struggling to start headful (?) Chrome to debug a test like I've done in the past. Might take some trial and error to figure that out.
[X] Test everything - New Chrome browser should be enabled everywhere in the repository.
[x] Updating browsers - There's a script in Angular Dev Infra for updating browser version automatically which would be useful to fork as well.
[x] Refactor - I kept the existing directory structure and formatting conventions which definitely clashes with @rules_prerender. I'll want to refactor things to make it more maintainable and aligned with the rest of the codebase.
[x] Migrate to shadowrootmode - Once everything is stable, I should be able to migrate all usages of shadowroot to shadowrootmode.
[ ] Break @io_bazel_rules_webtesting dependency - If possible, I should also look into breaking the dependency on @io_bazel_rules_webtesting entirely. Unfortunately the Angular implementation does still rely on a few exported symbols from that ruleset, even if it versions its own browsers. It would be great to drop that dependency altogether.

I've made a lot of progress here and landed locally versioned browsers. Chromium and Firefox are versioned locally and run for all e2e tests by default successfully. I then was able to update <template shadowroot="open"> -> <template shadowrootmode="open"> to fix the main issue. However there are two tasks I've failed at so far.

First, Firefox works great locally, but I have yet to get it to successfully run in CI. I'm not entirely sure what's going on as I've whack-a-moled a few errors. The one I'm at now is:

[GFX1-]: glxtest: Unable to open a connection to the X server
[GFX1-]: glxtest: libEGL initialize failed
[GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt

I don't fully understand what this means, but libegl doesn't seem to be configured correctly. I tried installing it through various means with no luck. eglinfo gives no so helpful output:

NEEDRESTART-VER: 3.5
NEEDRESTART-KCUR: 5.15.0-[104](https://github.com/dgp1130/rules_prerender/actions/runs/5703331116/job/15455875219#step:3:105)1-azure
NEEDRESTART-KEXP: 5.15.0-1041-azure
NEEDRESTART-KSTA: 1
libEGL warning: failed to open /dev/dri/card0: Permission denied
EGL client extensions string:
    EGL_EXT_device_base EGL_EXT_device_enumeration EGL_EXT_device_query
    EGL_EXT_platform_base EGL_KHR_client_get_all_proc_addresses
    EGL_EXT_client_extensions EGL_KHR_debug EGL_EXT_platform_device
    EGL_EXT_platform_wayland EGL_KHR_platform_wayland
    EGL_EXT_platform_x11 EGL_KHR_platform_x11 EGL_EXT_platform_xcb
    EGL_MESA_platform_gbm EGL_KHR_platform_gbm
    EGL_MESA_platform_surfaceless

GBM platform:
eglinfo: eglInitialize failed

Wayland platform:
eglinfo: eglInitialize failed

X11 platform:
eglinfo: eglInitialize failed

libEGL warning: failed to open /dev/dri/card0: Permission denied
Device platform:
eglinfo: eglInitialize failed

My takeaway is that it's failing to initialize, but I have no idea why or what to do to fix it. Resources on this are unfortunately pretty scarce. libegl seems to be related to 3D hardware acceleration, so I looked into turning that off altogether for Firefox. This appears to be possible, but only in the settings UI. I couldn't find a flag to do it.

For now, I have Firefox tests running with the noci tag. That at least means I'll see test failures locally with bazel test //..., even if they don't show in CI. My current best attempt is in the firefox branch, but I think I need to take a break on this for a while and come back later. I filed https://github.com/dgp1130/rules_prerender/issues/75 to follow up on this at some point.

Second I wasn't able to break the dependency on @io_bazel_rules_webtesting. I was hoping this wouldn't be too bad, but there is a significant amount of code there doing real work I can't drop. I initially forked the repo into @rules_prerender, linked it locally with local_repository and then started deleting everything I could. I was hoping it was just build rules, but the repository has a non-trivial amount of Go infrastructure it needs to even start a web_test_suite test. I don't see a local fork as an advantage over an unmaintained dependency in this case, so I'd rather just leave the dep as is. It's a bit unfortunate, but it's the least bad option right now. My attempt is in ref/rm-rules-webtesting, but I don't think I'll be moving forward with it.

For now, I'll close this issue and follow up on the Firefox thing separately.

dgp1130 / rules_prerender

Manually version browsers #74