Upgrading Xcode version from 15.2 to 16.0 and MacOS version from 13 to 14 results in CI/CD simulator architecture errors

tonioleetchi commented 3 weeks ago

Description

I have the the same issues like this post : https://github.com/actions/runner-images/issues/10104. So, I will repeat the majority of the information in this ticket. We have been running our unit tests and releases through our Azure DevOps CI/CD pipeline for some time now, and everything is going well. We have now decided to upgrade our current settings to the new runner as well as the new version of Xcode

Working settings before upgrade:

- name: runnerImage
  value: 'macOS-13'
- name: XcodeVersion
  value: 'Xcode_15.2.app'
- name: iOSSimulator
  value: 'iPhone 15 Pro,OS=17.2'

Failing settings after upgrade:

- name: runnerImage
  value: 'macOS-14'
- name: XcodeVersion
  value: 'Xcode_16.0.app'
- name: iOSSimulator
  value: 'iPhone 16 Pro,OS=18.0'

For information, our project compiles well locally with Xcode16 on a Mac M1Pro and on an Intel Mac

Error message

xcodebuild: error: Failed to build workspace XXX with scheme XXX Debug.: Cannot test target XXXTests on iPhone 16 Pro: XXXTests does not support any of iPhone 16 Pro’s architectures: x86_64

Platforms affected

[X] Azure DevOps
[ ] GitHub Actions - Standard Runners
[ ] GitHub Actions - Larger Runners

Runner images affected

[ ] Ubuntu 20.04
[ ] Ubuntu 22.04
[ ] Ubuntu 24.04
[ ] macOS 12
[X] macOS 13
[X] macOS 13 Arm64
[X] macOS 14
[X] macOS 14 Arm64
[ ] Windows Server 2019
[ ] Windows Server 2022

Image version and build link

Agent machine name: 'Mac-1727208462714' Current image version: '20240923.101'

Is it regression?

Yes, same runner image though. The failure started upgrading the app from Xcode 15.2 to 16.0. We noticed that there is the same issue on Xcode 15.4

Expected behavior

Tests succeeding like they did on Xcode_15.2.app, where the x86_64 architecture for iOS Simulator is not an issue.

Actual behavior

Builds are failing on Xcode_16.0.app where the x86_64 architecture for iOS Simulator is an issue.

Repro steps

Exeucted command:

usr/bin/xcodebuild -sdk iphoneos -configuration Debug -workspace /Users/runner/work/1/s/XXX.xcodeproj/project.xcworkspace -scheme XXX Debug -destination platform=iOS Simulator,name=iPhone 16 Pro,OS=18.0 test -archivePath /Users/runner/work/1/s/build/XXX Debug.xcarchive -verbose CODE_SIGNING_REQUIRED=Yes CODE_SIGNING_ALLOWED=No | /usr/local/lib/ruby/gems/3.0.0/bin/xcpretty -r junit --no-color

erik-bershel commented 3 weeks ago

Hey @tonioleetchi! Have you tried running the build on the macOS-14 image with Xcode 15.2? If not, please try. It also exists in the image. We would also really appreciate an example project, even a very simple one, for testing the problem.

tonioleetchi commented 3 weeks ago

Hey @erik-bershel , works fine on macOS-14 image and Xcode 15.2! But on macOS-14 image and Xcode 15.3, I have the same issue like macOS-14 image and Xcode 16.0. Ok, I will try to provide you a simple iOS project to make tests.

erik-bershel commented 3 weeks ago

A couple more clarifying questions. 🤔

You say that in local environments the compilation process completes correctly for both Apple Silicon and Intel. Do I understand correctly that you meant both compilation on the simulator and testing using the simulator?

Also, could you provide the scheme? I'd like to see what devices it's targeting.

I'm not entirely sure what targets are used by default during compilation and what targets the iPhone 16 / iOS 18 simulator supports. There is a very old thread that goes into some detail for similar cases from the era when arm64 was just emerging as a desktop solution: stackoverflow.com/questions/63391793.

Also, it would be great to try to build a project for such a matrix (this way we can localise the problem with the device itself):

- name: runnerImage
  value: 'macOS-14'
- name: XcodeVersion
  value: 'Xcode_16.0.app'
- name: iOSSimulator
  value: 'iPhone 15 Pro,OS=17.2'

You can also check the settings of supported architectures:

xcodebuild -workspace <YourWorkspace>.xcworkspace -scheme <YourScheme> -showBuildSettings

There should be something like:

ARCHS = arm64 x86_64
VALID_ARCHS = arm64 x86_64

tonioleetchi commented 3 weeks ago

Thanks for the info. I made the same changes as in this post: https://stackoverflow.com/a/64150387 , it fixed the problem.

Here's what changed:

Without any modications in in my main target build settings:

Executed command:

xcodebuild -workspace XXX.xcodeproj/project.xcworkspace -scheme "XXX Debug" -showBuildSettings

Result:

HOST_ARCH = x86_64
VALID_ARCHS = arm64 arm64e armv7 armv7s
arch = undefined_arch

After adding VALID_ARCHS = $(ARCHS_STANDARD) in User-Defined in my main target build settings :

Executed command:

xcodebuild -workspace XXX.xcodeproj/project.xcworkspace -scheme "XXX Debug" -showBuildSettings

Result:

HOST_ARCH = x86_64
VALID_ARCHS = arm64
arch = undefined_arch

Surprisingly this modification seems to have corrected the problem, I no longer have any problems on my CI/CD.

Thanks a lot for your help @vhiairrassary

erik-bershel commented 3 weeks ago

Hey @tonioleetchi! I am glad you figured it out. 🚀

Closing issue since:

Surprisingly this modification seems to have corrected the problem, I no longer have any problems on my CI/CD.

🎉 🎉 🎉

actions / runner-images