Should we support GN as an official build system?

[sffc]

GN is a build system popular with various open-source projects such as V8, Fuchsia, and Skia, all of which are potential ICU4X adopters (and all of which happen to have originated at Google).

I think we should consider supporting GN as an official build system for ICU4X, with a focus on FFI. I would like to lay out my reasoning.

Lemma 1: We should prioritize ease of adoption.

Programmers are busy. Because of that, programmers are more likely to adopt a solution that is easy for them to adopt. Because of that, solutions with easy onboarding/adoption will get higher adoption. Because of that, and because ICU4X wants to get more clients, ease of adoption should be a priority for us, especially as we scale beyond the first few early adopters.

Lemma 2: For many ICU4X clients, the primary adoption blocker is Rust.

This is a bold statement, until you realize that most of our clients for the foreseeable future are going to be in non-Rust languages, especially C/C++. At least at Google, we are often the first Rust code in our clients' code bases. Some potential customers won't even speak to us out of concern that Rust will cause integration headaches for them.

Lemma 3: We can ease adoption by shipping build files for popular C++ build systems.

Build systems like GN support Rust fairly well, making it easy to use from C++ programs, but this isn't immediately evident to customers. I personally spent several evenings last week fiddling with various parts of the GN ecosystem to get ICU4X up and running. Having pre-built GN configuration files would have made it much easier.

Lemma 4: It is not uncommon for projects to support multiple official build systems.

Our parent library, ICU4C, supports two (autoconf and vcxproj), and plans to add more (cmake, bazel, and/or meson). A quick Google search can turn up more.

---

Now, the obvious counter-argument is, "GN is a Google thing! We shouldn't force all ICU4X contributors to maintain Google-specific build files." This is a valid argument. Here are my counter-points:

• GN is open-source and anyone can use it.
• GN is special because it is used by multiple potential customers.
• GN sets the stage for supporting more build systems in the future.

That last point is key. I'm suggesting that we consider GN as the first non-Cargo build system for ICU4X, but I would like this to set the stage for how we support more such build systems in the future. We should think about where we store the configuration files, how we generate them, and how we keep them up-to-date.

Discussion questions:

1. Should the ICU4X open-source project support non-Cargo build systems?
2. If yes, should we support GN?
3. Should we add the GN files in the main repo (perhaps under /ffi) or elsewhere?
4. Should we add CI checks for keeping the GN files up-to-date, or should this be done at release time? (Note: the files should generally need to be updated only when a PR adds a crate or a new dependency; basically whenever the lockfile changes.)

Needs input from:

• [x] @zbraniecki
• [x] @Manishearth
• [x] @dminor
• [x] @gregtatum
• [x] @nciric

[Manishearth]

I think we should definitely try to have this.

1. Should the ICU4X open-source project support non-Cargo build systems?

Yes, though perhaps as second-class: some CI breakage might be allowed depending on how good our tooling is (but perhaps not allowed to make it to a release)

2. If yes, should we support GN?

Yes

3. Should we add the GN files in the main repo (perhaps under /ffi) or elsewhere?

My main worry is having this be a burden on people who are not touching FFI, if regenerating GN is mostly a single cargo-make command I think it's fine to have it in the repo, though I think at least to start with we should perhaps have it in a separate repo.

An option is to have the GN configs in tree but have a separate repo for the generated build files.

4. Should we add CI checks for keeping the GN files up-to-date, or should this be done at release time? (Note: the files should generally need to be updated only when a PR adds a crate or a new dependency; basically whenever the lockfile changes.)

If it's in tree it should be checked by CI. If we follow the configs-only approach mentioned above, we can have a separate repo with its own CI that we periodically update with generated files.

[gregtatum]

In principle I'm open to having the ICU4X team support the build files. We should absolutely lower friction to new adopters. It will help with the health and long term viability of the project to have more users. I'm hoping Mozilla will have a happy path for loading the FFI, and Google projects should as well.

My questions would be around maintenance cost, and how much more complicated releases will be. Before officially adding them, it would be nice to get a prototype done of what this would look like so we can understand the maintenance cost and risks.

Should the ICU4X open-source project support non-Cargo build systems?

Yes, we should have a principle of making it easy for our users to adopt ICU4X.

If yes, should we support GN?

It seems reasonable to explore the cost/benefits, see my points above prototyping.

Should we add the GN files in the main repo (perhaps under /ffi) or elsewhere?

Config files seem clear to add, while I think we should explore where the generated files live. I agree with Manish's points on there is a risk of making day-to-day work harder to maintain. I'm not sure of the trade off between releases being difficult to handle breaking changes for the FFI vs breaking changes in individual PRs.

I have a general heuristic that mono repos are easier to maintain and coordinate than multiple ones. I would say if we can get the steps down to a single reasonable command it might be nice to have it all in one repo.

Should we add CI checks for keeping the GN files up-to-date, or should this be done at release time? (Note: the files should generally need to be updated only when a PR adds a crate or a new dependency; basically whenever the lockfile changes.)

I'm generally in favor of CI making sure we do the right things as long as they are not flaky. Here again a prototype would be good to understand the flow. I haven't personally messed around with gn so do not understand the actual workflow we would be committing ourselves to.

I would be curious to hear the other Mozillian's thoughts for this as well.

[nciric]

It seems that we have an agreement to move forward with a prototype, potentially starting with gn as initial candidate.

Let's try it out and see the costs. I would assume we can auto-gen some of the files to lower the burden.

[dminor]

I've worked with gn in the context of the webrtc.org project. My main concern about gn is that the syntax is not stable and evolves over time, and there does not seem to be any release numbering scheme that allows us to track which version we're using in the project. Google projects just pin a commit of gn to use, which I guess is the approach we can take here.

I think we should support other build systems and it is worth prototyping with gn, but I am a little bit concerned about the maintenance burden if we need developers to update configs for a bunch of different build systems.