macos_unit_test CPU Architecture Selection

tinder-cwybranowski commented 3 years ago

Hello! I've been using macos_unit_test in my projects, and I recently noticed that it does not respect the macos_cpus configuration flag when invoked via bazelisk test. Although this has no impact on my Intel machine, my Apple Silicon machine is unable to invoke tests using arm64.

Setup

Bazel 4.2.0
Xcode 12.5.1
rules_apple version 0.31.3

Demo Repository

I've created a public repository that demonstrates this issue: rules-apple-demo.

Everything should be detailed in the README of that repository, but I'll post some information below for visibility.

Issue

As described in the demo repository (mentioned above), the test output of the bazelisk test invocation indicates that the underlying xcodebuild invocation used the x86_64 architecture, despite explicitly passing arm64:

Command line invocation:
    /Applications/Xcode12.5.1.app/Contents/Developer/usr/bin/xcodebuild test-without-building -destination platform=macOS,arch=x86_64 -xctestrun /var/folders/d2/26js2mt159n948c9ry6mt60c0000gp/T//test_tmp_dir.KXf4bB/tests.xctestrun

keith commented 3 years ago

I believe this was fixed by https://github.com/bazelbuild/rules_apple/pull/1191 which isn't in a release yet, can you test and see if it fixes your use case?

tinder-cwybranowski commented 3 years ago

I appreciate the quick reply, @keith! I just tried with revision 0a2c39c0209087e39818c16908c545d744a2e0ca and it does seem to work now 🎉

I did notice that the make success invocation now fails with an issue similar to the original -- seems like x86_64 isn't making it to the underlying xcodebuild invocation:

Command line invocation:
    /Applications/Xcode12.5.1.app/Contents/Developer/usr/bin/xcodebuild test-without-building -destination platform=macOS -xctestrun /var/folders/d2/26js2mt159n948c9ry6mt60c0000gp/T//test_tmp_dir.A4eoYk/tests.xctestrun

keith commented 3 years ago

Hrm yea so the assumption we made in that change was that the binary can be built by bazel however it wants, respecting the CPUs flags you pass, and then when we run the tests it should "just work" without requiring passing the CPU again. But in this case it seems xcodebuild is reluctant to run x86_64 tests on an M1 host unless you pass the arch explicitly, which is kinda interesting. I guess that use case could make sense sometimes, but I'm not sure how big of a deal it is since I assume most of the time you're running macOS tests you'll be running them for the current host's arch?

Either way I think the potential fix here is to pass arch=the actual arch of the binary, which probably isn't a big deal but requires a bit of annoying output parsing

keith commented 3 years ago

It gets a bit more complicated with fat binaries, since I guess you have a few potential cases here:

on intel host, testing arm binary, fail
on m1 host, testing arm binary, fine with arch=arm64
on m1 host, testing intel binary, fine with arch=x86_64
on intel host, testing fat binary, fine with arch=x86_64
on m1 host, testing fat binary, fine with either arch, but should pass arch=arm64

tinder-cwybranowski commented 3 years ago

I appreciate the extra context, and that makes sense. It was an unexpected behavior, but also not sure how common this use-case is. The more important support (at least in my case) is that M1 can invoke bazelisk test on macos_unit_test targets, which it can as of 0a2c39c0209087e39818c16908c545d744a2e0ca 🙏

And on your note about fat binaries -- can test targets be bundled as fat binaries? And if so, what rule(s) facilitate that?

keith commented 3 years ago

I think if you built with --macos_cpus=x86_64,arm64 it might, but given that would be duplicate building I'm not sure you'd want to

tinder-cwybranowski commented 3 years ago

Ah okay, interesting. Well I appreciate the help addressing the core issue described here 😄

bazelbuild / rules_apple