Engine CI

[ardera]

If you're looking into building a github actions based engine CI, I have a repo here that might or might not be helpful: https://github.com/ardera/flutter-ci

I have that because linux armv7 is not officially supported by Flutter, so I need to build my own binaries. It builds:

• The linux armv7/aarch64/x64 libflutter_engine.so
  • In all runtime modes
  • Also including a special debug_unopt flavor, so you can (in combination with the dbgsyms) easily step through engine code.
  • All engines are built with debug symbols (-ggdb) by default, and the debug symbols are then split out into a separate file, so you don't necessarily need download & bundle them, but you can if you want to.
  • Also including pi3/pi4-tuned variants of the profile & release mode engines.
• gen_snapshots that run on Windows, macOS and Linux armv7/aarch64/x64 and target Linux armv7/aarch64/x64.
  • Required some patches, normally you can only build a gen_snapshot targeting Linux on Linux itself, and you also can't build a gen_snapshot that e.g. should run on aarch64, from an x64 build machine.

It all runs in the github free tier, the complete pipeline takes about 1h30min in my case, but only 1h of that is build time, the rest is some basic smoke testing I'm doing. @jezell

[jezell]

Awesome. I also have some internal scripts we've been working on for Web + iOS + Android. Github actions based is definitely the way we want to go so it makes it easy for others to setup their own copies and run internally as well. The runs are definitely long, but we can probably optimize that or use alternative runners if it becomes an issue. GH actions i/o tends to be really bad.

[jezell]

FYI: this is the setup that is coming soon, so we'll want compatible workflows

https://github.com/flutter/flaux/

[ardera]

Awesome. I also have some internal scripts we've been working on for Web + iOS + Android. Github actions based is definitely the way we want to go so it makes it easy for others to setup their own copies and run internally as well. The runs are definitely long, but we can probably optimize that or use alternative runners if it becomes an issue. GH actions i/o tends to be really bad.

True, the runs are a bit long. I wouldn't really know how to optimize it, but yeah I'd also say, if people want faster builds they can just hook up some self-hosted runners.

One thing I'd like to do differently though is, counter-intuitively, using GitHub Actions less :) Basically I want to move a lot of the logic to shell scripts and invoke those from the workflows. Sounds a bit hacky, but the problem is that you unfortunately can't test gh-actions workflow files locally. Right now, every time I change something in the package step for example, I push the change, do work on something else for 2 hours, and only then can I see if it works. With shell scripts you could test some things locally. I even heard that's how some teams at GH do it internally.

FYI: this is the setup that is coming soon, so we'll want compatible workflows

https://github.com/flutter/flaux/

Oh interesting, I've heard of the monorepo before, but I thought it was only meant for CI/testing for now. Guess that means I'll have to rework my flutter-ci soon. :)

[jezell]

Yeah, the pain of debugging github actions is real. There is a local runner though:

https://github.com/nektos/act

Might have to give this a shot. A lot of the flutter stuff is already wrapped in some pretty good tooling for build and test, but it can definitely take a really long time to debug simple problems when you have to commit and push, then wait 40 minutes to find out there is a problem.