Upgrade to XCode 16 - Githubissues

jwnimmer-tri commented 1 month ago

Is your feature request related to a problem? Please describe.

In #21529 and #22029, we discovered that the standard library in XCode 14 and 15 is missing some support for spaceship (<=>).

Describe the solution you'd like

Upgrade our macOS Sonoma CI image(s) to XCode 16.

When creating our macOS Sequoia CI image(s), use XCode 16.

Of course then also we need to update our docs to reflect current CI, e.g.:

Describe alternatives you've considered

Pollute our code with backcompat work-arounds.

Additional context

This is not an emergency. It can wait until we are already reworking the CI images.

In particular the until we do the #21910 Sequoia images and resize the images for #21977.

rpoyner-tri commented 1 month ago

FWIW, Xcode 15.3 seems to be at odds with updated bazel tool chain rules as well:

  /Users/ricopoyner/checkout/drake/WORKSPACE:12:19: in <toplevel>
  /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl:74:23: in apple_cc_configure
Repository rule _apple_cc_autoconf defined at:
  /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl:55:37: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_config_apple_cc':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl", line 49, column 49, in _apple_cc_autoconf_impl
                success, error = configure_osx_toolchain(repository_ctx)
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/osx_cc_configure.bzl", line 206, column 21, in configure_osx_toolchain
                _compile_cc_file(repository_ctx, libtool_check_unique_src_path, "libtool_check_unique")
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/osx_cc_configure.bzl", line 121, column 13, in _compile_cc_file
                fail(out_name + " failed to generate. Please file an issue at " +
Error in fail: libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: no such package '@@local_config_apple_cc//': libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: /Users/ricopoyner/checkout/drake/multibody/parsing/BUILD.bazel:272:17: //multibody/parsing:_detail_usd_parser_headers_cc_impl depends on @@local_config_apple_cc//:cc-compiler-darwin_arm64 in repository @@local_config_apple_cc which failed to fetch. no such package '@@local_config_apple_cc//': libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
Use --verbose_failures to see the command lines of failed build steps.
ERROR: Analysis of target '//multibody/parsing:_detail_usd_parser_headers_cc_impl' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.627s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
ERROR: Build did NOT complete successfully
ERROR: No test targets were found, yet testing was requested
FAILED: 
    Fetching repository @@mujoco_menagerie_internal; starting
    Fetching repository @@buildifier; starting
    Fetching repository @@xmlrunner_py; starting
    Fetching repository @@crate__lapack-sys-0.14.0; starting
    Fetching repository @@vtk_internal; starting
    Fetching repository @@dm_control_internal; starting
    Fetching /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/crate__lapack-sys-0.14.0; Extracting download.tar.gz
    Fetching repository @@uwebsockets_internal; starting
ricopoyner@macsim-m1-1 drake % bazel clean --expunge --async
INFO: Starting clean.
INFO: Output base moved to /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985_tmp_99902_e60368b1-798a-43a1-bb39-a5fd5f949c0e for deletion
ricopoyner@macsim-m1-1 drake % bazel test //multibody/parsing/...
Starting local Bazel server and connecting to it...
INFO: Repository local_config_apple_cc instantiated at:
  /Users/ricopoyner/checkout/drake/WORKSPACE:12:19: in <toplevel>
  /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl:74:23: in apple_cc_configure
Repository rule _apple_cc_autoconf defined at:
  /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl:55:37: in <toplevel>
ERROR: An error occurred during the fetch of repository 'local_config_apple_cc':
   Traceback (most recent call last):
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/setup.bzl", line 49, column 49, in _apple_cc_autoconf_impl
                success, error = configure_osx_toolchain(repository_ctx)
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/osx_cc_configure.bzl", line 206, column 21, in configure_osx_toolchain
                _compile_cc_file(repository_ctx, libtool_check_unique_src_path, "libtool_check_unique")
        File "/private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/osx_cc_configure.bzl", line 121, column 13, in _compile_cc_file
                fail(out_name + " failed to generate. Please file an issue at " +
Error in fail: libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: no such package '@@local_config_apple_cc//': libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: /Users/ricopoyner/checkout/drake/multibody/parsing/BUILD.bazel:104:17: //multibody/parsing:collision_filter_groups depends on @@local_config_apple_cc//:cc-compiler-darwin_arm64 in repository @@local_config_apple_cc which failed to fetch. no such package '@@local_config_apple_cc//': libtool_check_unique failed to generate. Please file an issue at https://github.com/bazelbuild/apple_support/issues with the following:
return code 1, stderr: /private/var/tmp/_bazel_ricopoyner/27b47a6d9b400570878eb2115555e985/external/build_bazel_apple_support/crosstool/libtool_check_unique.cc:15:10: fatal error: 'cstdlib' file not found
   15 | #include <cstdlib>
      |          ^~~~~~~~~
1 error generated.
, stdout: 
ERROR: Analysis of target '//multibody/parsing:collision_filter_groups' failed; build aborted: Analysis failed
INFO: Elapsed time: 65.308s, Critical Path: 0.84s
INFO: 33 processes: 1 internal, 32 darwin-sandbox.
ERROR: Build did NOT complete successfully

See on macos 14.7, xcode 15.3.

rpoyner-tri commented 1 month ago

I've got a local fleet of macs with macos 14.7, xcode 16. None of them can build drake without link-time failures. If I downgrade to 15.4 (currently), all is well.

I think we need to figure out how to have Drake live with xcode 16. It doesn't necessarily block this CI work. I'll file a separate issue to track the gory Drake details.

jwnimmer-tri commented 1 month ago

New(?) approach:

Keep Sonoma (i.e., our existing platform) on XCode 15.
For the new Sequoia images (i.e., the new platform) install XCode 16. That means that CI will surely fail due to #22061, but that's good. We can test PR fixes in CI.

williamjallen commented 2 weeks ago

The new Sequoia images discussed in https://github.com/RobotLocomotion/drake/issues/21910#issuecomment-2452365358 contain Xcode 16.

rpoyner-tri commented 1 week ago

I believe that when https://github.com/RobotLocomotion/drake/issues/22176 is closed, the way will be clear to upgrade to Xcode 16 on either or both supported OS versions.

rpoyner-tri commented 5 days ago

Not so fast. Still outstanding is a fix for dynamic_cast in processes with multiple shared libraries. Recent sequoia CI builds will show the failures. See https://github.com/RobotLocomotion/drake/issues/22204

RobotLocomotion / drake

Upgrade to XCode 16 #22031