Platform Commonalities: e.g. Apple, POSIX, etc.

[cpsauer]

Hi wonderful Bazel Platforms folks!

First and foremost, thanks for working to create something centralized and great here. It's so nice to have official support for modeling cross-platform code with platform-specific backends (among other goodies).

I'd like to float an idea for making official something I find myself rebuilding pretty much every time I use platforms.

Lots of platforms (err constraint_values) have considerable code in common. It can be nice to select on this commonality. Some examples are: Apple Platforms. POSIX/UNIX-compliant platforms. Etc.

It's not hard to model these (example below), but it'd be awesome to have them in the @platforms repo for everyone to use!

load("@bazel_skylib//lib:selects.bzl", "selects")
selects.config_setting_group(
    name = "Apple",
    visibility = ["//visibility:public"],
    match_any = ["@platforms//os:macos", "@platforms//os:ios", "@platforms//os:tvos", "@platforms//os:watchos"],
)

Happy to contribute of course--and would have made this a PR--except that I'm not sure quite how this repo is being automagically integrated into Bazel proper. Therefore I wasn't sure how to properly get skylib.

Thanks again, additionally for your consideration! Chris (ex-Googler)

[cpsauer]

@gregestren, is this the right place for issues like this, or should I have filed on the main Bazel repo? (Seemed like most of the ones here aren't getting replies.)

[gregestren]

Interesting (and sorry I missed this the first time around). I think this is the right repo to discuss this and my intuition is this could be a good fit. I'd like to discuss this with the configurability team so I'll add it to their sync agenda and report back after our next sync.

[cpsauer]

Thanks for being so great to work with, @gregestren :)

[gregestren]

We chatted about this (@katre @aranguyen @aiuto @sdtwigg ).

I'm honestly not sure what the right path is.

• We considered the alternative of making Apple a constraint_value on something like ecosystem, but thought it doesn't quite fit the platform idea and adds yet another type platforms and toolchains have to integrate
• We talked about having both this and the Mac-related OS constraints in the apple_rules repo instead of here. I think the argument was that, while this repo is intended to be universally accessible, anyone building Apple stuff is going to have the apple_rules repo loaded anyway
• People are cautious about adding new dependencies on skylib from common places. @aiuto can probably explain better if there are versioning / diamond dep issues to consider

All that said, I do want this repo to remain relevant and serve its purpose. So I'm throwing these ideas out there for further consideration from all.

[cpsauer]

Thanks for leading the discussion! (Always happy to hop into a GVC if useful.)

Thoughts back in case they're helpful:

• Definitely agree re not adding these as another independent type of constraint_setting; the examples above (POSIX, Apple) are instead naturally expressed as subsets of the OSs and aren't mutually exclusive.
  • Better to express them as sets, like above, since, e.g., macOS -> Apple & POSIX.
• IMO these sets belong here, rather than federated or in rules_apple, so long as the OS values are centralized here.
  • For example, it's perfectly fine to have a cc_library/cc_binary (like boost, for example) that would use them, but doesn't/shouldn't otherwise require rules_apple. The boost rules would be a heavy user, I'd think. That's one of the places I had to reimplement this logic (perhaps time number 4 or so), and what prompted this issue.
  • This is the same reasoning, I'd think, that \@platforms//os:macos lives here rather than rules_apple. More generally, isn't the general philosophy to try to have general, universally selectable platforms logic live here?
• Re skylib, I'd guessed wrong and figured you'd probably require that we use skylib. Not hard to remove the dependency and duplicate the (fairly simple) implementation.

So what do you think? Would you be open to having POSIX and Apple here, expressed as convenient sets of OSs that match on their constituents--if it didn't use skylib?

[cpsauer]

Another good use case, sparked by https://github.com/bazelbuild/bazel/issues/14982: CPUs, in addition to OSs, have properties in common. Feels like, e.g., ARM could, for example, match any ARM CPU via the same mechanism.

[aiuto]

I'm not convinced we want this yet. There is always the balance of utility vs. correctness as the world evolves. Let's use the example of Apple := match_any = ["@platforms//os:macos", "@platforms//os:ios", "@platforms//os:tvos", "@platforms//os:watchos"].

What does that actually do for people? Or, let's ask a different question, why would one select on "Apple"? Some hypothetical answers:

• To select CPU. Obviously not any more.
• To select APIs available. This would be a lie, different SDKs are available on each.
• To select a need to build the release on macos? We can do that with existing constraint mechanisms.

We could play the same thought experiment with posix. Is any system really strictly posix? What we care about are things like

• has_posix_threads
• has_posix_file_system_semantics
• uses_elf

Now, we could imaging that the linux platform has lots of constraints describing the fine grained behaviors that might be of interest. Toolchains could talk about what they require rather than something broad like OS.

[cpsauer]

Hmm, I think there's actually a great use case among those. I mean, there's a vast API surface in common across Apple OSs. They're all forks of each other, after all. And it's therefore quite common to have code/settings that are shared between Apple OSs (offering a common API) but not with other OSs. In boost, for example, Apple OSs are selected upon together far more than they're selected apart. Similarly, think about all the Bazel code shared for Apple platforms but not with other platforms!

Said differently: One would select on "Apple" in the (relatively common) case where you're targeting Apple platforms together and using the API they have in common. (Maybe there are other use cases, but that's the main one I see.)

It's a great clarifying example, though. You're totally right that it's really about selecting on common APIs supported across OSs (or CPUs). In set semantics, the select matches a union of constraint_values, selecting for the intersection of the interfaces they provide. And yeah, whether the OS supports pthread is another one of these OS commonalities that gets selected on and reimplemented over and over.

Anyway, just think this could be handy and help people easily express the idea of a common platform!

[keith]

For some practical examples about the Apple use case envoy has quite a few of these types of conditionals https://cs.github.com/envoyproxy/envoy?q=%22bazel%3Aapple%22

[gregestren]

@aiuto what do you think about a skylib dependency from this repo?

[cpsauer]

@gregestren &co: Any chance I could return to this and ask whether you'd take a contribution for :Apple--and if so, whether you'd like a dependency on skylib or not?

The context is that I'm shipping another repo that needs this, and I'd love to contribute it rather than duplicating.

[keith]

Since this conversation we have added platform definitions in rules_apple https://github.com/bazelbuild/apple_support/blob/master/platforms/BUILD, and I think they might live there forever, I wonder if we should include those there instead? The only downside I can think of is the inconsistency of having to know when to uses @platforms or not. Assuming everyone who may need this already depends on that repo (potentially transitively). I would be happy to add more similar conditions there as needed

[cpsauer]

Hey, @keith! Good thinking--and thanks for making me aware of those new definitions.

I think, though, that there's a sizable use case for these platform commonalities (including the os:Apple one) where people wouldn't have brought in rules_apple already, and where I'm not sure we should require them to. rules_boost for the C++ boost libraries is a concrete example (as above), as is the repo I was just releasing, but more generally, fairly frequently, pure cc_libraries need to condition by platform to get the right set of syscalls. Those decencies might eventually get bundled up using Apple's conventions (or not), but they aren't currently bringing in rules_apple.

Therefore, it still seems to me like these platform commonalities should go here for the same reason (I'm guessing) the OSs are defined here: So you can be aware of them and select based on them way up the dependency chain.

[cpsauer]

Oh, and semi-relatedly, since it came up in rules_boost and in that repo I'm spinning up (https://github.com/hedronvision/bazel-make-cc-https-easy): The best story around using platforms with iOS and Android is still manual platform mappings, right? There isn't a way to magically auto patch that in via apple_support or similar?

[keith]

Yes you have to use platform_mappings, theoretically we could provide a default-ish platform_mappings file probably

[cpsauer]

Just got the commit bit on rules_boost, so I can now say a little less theoretically: We'd use the heck out of this in rules_boost--as heavily as any individual OS--and wouldn't otherwise want/need to depend on rules_apple for our non-apple users.

[gregestren]

At the moment I think we're being too cautious with @bazelbuild/platforms. We're rightfully concerned about it getting messy and disorganized. And any mistakes we make here can get so entrenched in the wider build ecosystem that it's impossible to undo them.

On the other hand, if it's not expansive enough people will just work around it by putting commonalities in arbitrary repos. Which is the exact problem this repo exists to avoid.

For :apple in particular I'd lean more heavily on @keith's intuition since rules_apple isn't an "arbitrary" repo.

[cpsauer]

Was just contributing to boringssl. These would be useful to have there (and not in rules_apple) for all the same reasons as boost.

[brentleyjones]

and not in rules_apple

I agree, rules_apple is a heavy dependency, but I think _applesupport is a super light one that makes sense given that the Xcode toolchain is moving there: https://github.com/bazelbuild/bazel/pull/16619

[cpsauer]

Offhand, my take would still be that these would still be best here alongside :macos, :ios, etc.--for the same reason I presume the apple platform definitions are here: That they're useful for pure C/C++ (posix) libraries and it's nice to have them all in one place. (Though, perhaps all, including os definitions, should move to apple_support? Depends on y'all's federation philosophy.) [Brought back here by needing to duplicate this code a couple more times in the interim.]