Setup CI

[buffrr]

We're gonna need some CI infra to do builds for Beacon this should make things a lot easier. Chromium uses LUCI it's a bit too googly i'm not sure if it's documented well enough to be usable for non-google projects

[mdaniel]

There's also luci-go but unknown if they're equivalent. Poking around in that infra tree surfaced what seems a lot more chromium-specific tool: https://chromium.googlesource.com/infra/goma/server/+/refs/tags/v0.0.25#goma-server This is the first I've heard of LUCI, but when I saw the architecture, I thought of this: https://blog.nelhage.com/post/building-llvm-in-90s/

I was actually wondering how one caches 33GB worth of chromium .git inside of GHA; I believe brave uses their own runners which are able to have the chromium repo mounted on EBS volumes or such. If you do go with some kind of lambda-esque approach, they now offer EFS mounts

:question: Have you considered contacting GitHub support to see if they have any suggestions?

Before I saw this ticket, I was actually going to propose creating a Vagrantfile, or a Dockerfile to at least allow contributors to be on the right versions of things

[buffrr]

There's also luci-go but unknown if they're equivalent.

I saw that too maybe worth looking into. It appears that they used buildbot before but switched to luci.

I believe brave uses their own runners which are able to have the chromium repo mounted on EBS volumes or such. If you do go with some kind of lambda-esque approach, they now offer EFS mounts

Ah I see so brave uses jenkins for their CI. Maybe goma+gha combo could work that would make things simpler.

Poking around in that infra tree surfaced what seems a lot more chromium-specific tool: https://chromium.googlesource.com/infra/goma/server/+/refs/tags/v0.0.25#goma-server

So goma can be used to off load compilation into cloud servers that's pretty neat.

To set this up it seems that we need

1. A service that implements a Remote Execution API apparently Brave uses EngFlow
2. A goma server that connects to such service.
3. and builds would use the goma client

I wonder if Github actions could use the goma client to off load builds so we can just setup a goma server/service to get something like CI with GHA hmm.

Have you considered contacting GitHub support to see if they have any suggestions?

I will send them an email maybe they know something.

[mdaniel]

I was curious how Arch did it, and it seems they don't use gclient et al, but rather curl https://commondatastorage.googleapis.com/chromium-browser-official/chromium-101.0.4951.41.tar.xz which is still hefty at 1.4G but more manageable (same for ubuntu)

Given that the GHA cache is limited to 10G, I was curious if there was a way of converting the tar to a "working" src that the build could use, but it seems to be missing at least llvm-build, possibly more

It did get pretty far, so it may be worth exploring how to sideload the llvm-build directory:

2022/04/29 02:54:07 [WARNING] environment variable `BEACON_CC_WRAPPER` is not set.
2022/04/29 02:54:07 [WARNING] environment variable `BEACON_GOOGLE_API_KEY` is not set.
2022/04/29 02:54:07 [WARNING] environment variable `BEACON_GOOGLE_DEFAULT_CLIENT_ID` is not set.
2022/04/29 02:54:07 [WARNING] environment variable `BEACON_GOOGLE_DEFAULT_CLIENT_SECRET` is not set.
Setting up the build
Done. Made 17269 targets from 3011 files in 13885ms
Building Beacon
ninja: Entering directory `out/Release'
[1/51787] COPY ../../beacon/components/welcome/beacon_welcome_proxy.ts gen/beacon/components/welcome/beacon_welcome_proxy.ts
[2/51787] CXX obj/buildtools/third_party/libc++/libc++/future.o
FAILED: obj/buildtools/third_party/libc++/libc++/future.o

where :point_up: is due to ccache: error: execv of ../../third_party/llvm-build/Release+Asserts/bin/clang++ failed: No such file or directory

[mdaniel]

I also just realized they have chromium GitHub mirror so if GitHub can work with you in some way, they don't need to pay for that clone traffic from GCP; that is, conceptually you just want a pipeline that does - uses: actions/checkout twice, once for chromium and once for Beacon. I can only presume gclient is bright enough to use the different origin, otherwise what'd be the point of having a GH mirror?

Related to that, I would be super curious if the Microsoft/git fork would work in CI. They allege it is for monorepos, which Beacon kind of is given its sibling dependency upon chromium, and that if a build doesn't touch every file the sparse checkout could be a huge win, too

[buffrr]

Just an update: I had a call with EngFlow and their current offering is more suitable for larger teams. They seem to be flexible with their plans so this won't be completely ruled out for now. Buildbuddy said their remote execution API might work with goma so I will check that out. Even if we have a CI just for linux that would be good enough as a start.

I was curious how Arch did it, and it seems they don't use gclient et al, but rather curl .... where point_up is due to ccache: error: execv of ../../third_party/llvm-build/Release+Asserts/bin/clang++ failed: No such file or directory

I see my understanding is that gclient also pulls those dependencies so perhaps that was the issue. I found a PR that builds Ungoogled chromium using Github actions on macOS which is interesting https://github.com/ungoogled-software/ungoogled-chromium-macos/pull/54 seems to take 11 to 13 hours :-D