open-mpi / ompi

Open MPI main development repository
https://www.open-mpi.org
Other
2.12k stars 857 forks source link

Configure error with command line tools from Xcode15. #11935

Closed cianciosa closed 11 months ago

cianciosa commented 11 months ago

Thank you for taking the time to submit an issue!

Background information

After updating to Xcode 15, I can no longer configure the build. When attempting to build from source the configure command fails with the following error.

checking prefix for global symbol labels... none
configure: error: Could not determine global symbol label prefix

What version of Open MPI are you using? (e.g., v3.0.5, v4.0.2, git branch name and hash, etc.)

openmpi-4.1.5

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Build from source tarball.

If you are building/installing from a git clone, please copy-n-paste the output from git submodule status.

Please describe the system on which you are running


Details of the problem

Full output of the configure command is

 % ./configure --enable-mpi-fortran
checking for perl... perl

============================================================================
== Configuring Open MPI
============================================================================

*** Startup tests
checking build system type... aarch64-apple-darwin22.6.0
checking host system type... aarch64-apple-darwin22.6.0
checking target system type... aarch64-apple-darwin22.6.0
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... config/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking whether make supports nested variables... yes
checking whether UID '61509548' is supported by ustar format... no
checking whether GID '1551083765' is supported by ustar format... no
checking how to create a ustar tar archive... none
checking dependency style of gcc... gcc3
checking whether make supports nested variables... (cached) yes
checking directory of build tree... /Users/m4c/Libraries/openmpi-4.1.5
checking directory of source tree... .
checking directory of prefix... NONE

*** Checking versions
checking for repo version... v4.1.5
checking Open MPI version... 4.1.5
checking Open MPI release date... Feb 23, 2023
checking Open MPI repository version... v4.1.5
checking for repo version... v4.1.5
checking Open MPI Run-Time Environment version... 4.1.5
checking Open MPI Run-Time Environment release date... Feb 23, 2023
checking Open MPI Run-Time Environment repository version... v4.1.5
checking for repo version... v4.1.5
checking Open SHMEM version... 4.1.5
checking Open SHMEM release date... Feb 23, 2023
checking Open SHMEM repository version... v4.1.5
checking for repo version... v4.1.5
checking Open Portable Access Layer version... 4.1.5
checking Open Portable Access Layer release date... Feb 23, 2023
checking Open Portable Access Layer repository version... v4.1.5
checking for bootstrap Autoconf version... 2.69
checking for bootstrap Automake version... 1.15
checking for boostrap Libtool version... 2.4.6

*** Initialization, setup
configure: builddir: /Users/m4c/Libraries/openmpi-4.1.5
configure: srcdir: /Users/m4c/Libraries/openmpi-4.1.5
checking for prefix by checking for opal_clean... no
installing to directory "/usr/local"

*** OPAL Configuration options
checking if want to run code coverage... no
checking if want to compile with branch probabilities... no
checking if want to debug memory usage... no
checking if want to profile memory usage... no
checking if want developer-level compiler pickyness... no
checking if want developer-level debugging code... no
checking if want to developer-level timing framework... no
checking if want to install project-internal header files... no
checking if want pretty-print stacktrace... yes
checking if want pty support... yes
checking if want weak symbol support... yes
checking if want dlopen support... yes
checking for default value of mca_base_component_show_load_errors... enabled by default
checking if want heterogeneous support... no
checking if word-sized integers must be word-size aligned... no
checking if want IPv6 support... no
checking if want package/brand string... Open MPI m4c@MAC139088 Distribution
checking if want ident string... 4.1.5
checking if want to use an alternative checksum algo for messages... no
checking maximum length of processor name... 256
checking maximum length of error string... 256
checking maximum length of object name... 64
checking maximum length of info key... 36
checking maximum length of info val... 256
checking maximum length of port name... 1024
checking maximum length of datarep string... 128
checking if want getpwuid support... yes
checking for zlib in... (default search paths)
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
looking for library without search path
checking for library containing deflate... -lz
checking will zlib support be built... yes
checking __NetBSD__... no
checking __FreeBSD__... no
checking __OpenBSD__... no
checking __DragonFly__... no
checking __386BSD__... no
checking __bsdi__... no
checking __APPLE__... yes
checking __linux__... no
checking __sun__... no
checking __sun... no
checking netdb.h usability... yes
checking netdb.h presence... yes
checking for netdb.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking netinet/tcp.h usability... yes
checking netinet/tcp.h presence... yes
checking for netinet/tcp.h... yes
checking for struct sockaddr_in... yes
checking if --with-cuda is set... not set (--with-cuda=)
checking whether CU_POINTER_ATTRIBUTE_SYNC_MEMOPS is declared... no
checking whether cuPointerGetAttributes is declared... no
checking if have cuda support... no
checking if user requested PMI support... no
checking if user requested internal PMIx support()... no
checking for pmix.h in /usr... not found
checking for pmix.h in /usr/include... not found
configure: WARNING: discovered external PMIx version is less than internal version 3.x
configure: WARNING: using internal PMIx

*** ORTE Configuration options
checking if want orterun "--prefix" behavior to be enabled by default... no

*** OMPI Configuration options
checking if want compile-time warnings inside of mpi.h... yes
checking if want sparse process groups... no
checking if want peruse support... no
checking if want Fortran MPI bindings... yes (default)
checking if want C++ bindings... no
checking if want MPI::SEEK_SET support... yes
checking if want run-time MPI parameter checking... runtime

*** OSHMEM Configuration options
checking if want oshmem... not supported on this platform
checking if want SGI/Quadrics compatibility mode... yes
checking if want OSHMEM API parameter checking... always
checking for on_exit... no
checking if want pshmem... yes
checking if want to build OSHMEM fortran bindings... yes
no
checking if want custom libmpi(_FOO) name... mpi
checking if want wrapper compiler rpath support... yes
checking if want wrapper compiler runpath support... yes

============================================================================
== Compiler and preprocessor tests
============================================================================

*** C compiler and preprocessor
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking whether gcc understands -c and -o together... (cached) yes
checking if gcc requires a flag for C11... no
configure: verifying gcc supports C11 without a flag
checking if gcc  supports C11 _Thread_local... yes
checking if gcc  supports C11 atomic variables... yes
checking if gcc  supports C11 _Atomic keyword... yes
checking if gcc  supports C11 _Generic keyword... yes
checking if gcc  supports C11 _Static_assert... yes
checking if gcc  supports C11 atomic_fetch_xor_explicit... yes
configure: no flag required for C11 support
checking if gcc  supports __thread... yes
checking if gcc  supports C11 _Thread_local... yes
checking for the C compiler vendor... gnu
checking for ANSI C header files... (cached) yes
checking if gcc supports -finline-functions... yes
checking if gcc supports -fno-strict-aliasing... yes
configure: WARNING:  -fno-strict-aliasing has been added to CFLAGS
checking if gcc supports __builtin_expect... yes
checking if gcc supports __builtin_prefetch... yes
checking if gcc supports __builtin_clz... yes
checking for C optimization flags... -O3 -DNDEBUG -finline-functions -fno-strict-aliasing
checking for Interix environment... no
checking for C ident string support... static const char
checking for int8_t... yes
checking for uint8_t... yes
checking for int16_t... yes
checking for uint16_t... yes
checking for int32_t... yes
checking for uint32_t... yes
checking for int64_t... yes
checking for uint64_t... yes
checking for int128_t... no
checking for __int128... yes
checking for uint128_t... no
checking for long long... yes
checking for __float128... no
checking for long double... yes
checking complex.h usability... yes
checking complex.h presence... yes
checking for complex.h... yes
checking for float _Complex... yes
checking for double _Complex... yes
checking for long double _Complex... yes
checking for intptr_t... yes
checking for uintptr_t... yes
checking for mode_t... yes
checking for ssize_t... yes
checking for ptrdiff_t... yes
checking size of _Bool... 1
checking size of char... 1
checking size of short... 2
checking size of int... 4
checking size of long... 8
checking size of long long... 8
checking size of float... 4
checking size of double... 8
checking size of long double... 8
checking size of float _Complex... 8
checking size of double _Complex... 16
checking size of long double _Complex... 16
checking size of void *... 8
checking size of size_t... 8
checking size of ssize_t... 8
checking size of ptrdiff_t... 8
checking size of wchar_t... 4
checking size of pid_t... 4
checking size of atomic_short... 2
checking size of atomic_int... 4
checking size of atomic_long... 8
checking size of atomic_llong... 8
checking alignment of bool... 1
checking alignment of int8_t... 1
checking alignment of int16_t... 2
checking alignment of int32_t... 4
checking alignment of int64_t... 8
checking alignment of char... 1
checking alignment of short... 2
checking alignment of wchar_t... 4
checking alignment of int... 4
checking alignment of long... 8
checking alignment of long long... 8
checking alignment of float... 4
checking alignment of double... 8
checking alignment of long double... 8
checking alignment of float _Complex... 4
checking alignment of double _Complex... 8
checking alignment of long double _Complex... 8
checking alignment of void *... 8
checking alignment of size_t... 8
checking for weak symbol support... no
checking for macro weak symbol support... no
checking for functional offsetof macro... yes

*** C++ compiler and preprocessor
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking how to run the C++ preprocessor... g++ -E
checking for the C++ compiler vendor... gnu
checking if g++ supports -finline-functions... yes
configure: WARNING:  -finline-functions has been added to CXXFLAGS
checking if C and C++ are link compatible... yes
checking for C++ optimization flags... -O3 -DNDEBUG -finline-functions
checking size of bool... 1
checking alignment of bool... (cached) 1

*** C++ compiler and preprocessor
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking dependency style of g++... (cached) gcc3
checking how to run the C++ preprocessor... g++ -E
checking if C++ compiler works... yes
checking if g++ supports -finline-functions... yes
configure: WARNING:  -finline-functions has been added to CXXFLAGS
checking if C and C++ are link compatible... (cached) yes
checking for C++ optimization flags... -O3 -DNDEBUG -finline-functions
checking size of bool... (cached) 1
checking alignment of bool... (cached) 1
checking if able to build the MPI C++ bindings... no
checking if want C++ exception handling... skipped

*** Compiler characteristics
checking for __attribute__... yes
checking for __attribute__(aligned)... yes
checking for __attribute__(always_inline)... yes
checking for __attribute__(cold)... yes
checking for __attribute__(const)... yes
checking for __attribute__(deprecated)... yes
checking for __attribute__(deprecated_argument)... yes
checking for __attribute__(error)... yes
checking for __attribute__(format)... yes
checking for __attribute__(format_funcptr)... no
checking for __attribute__(hot)... yes
checking for __attribute__(malloc)... yes
checking for __attribute__(may_alias)... yes
checking for __attribute__(no_instrument_function)... yes
checking for __attribute__(noinline)... yes
checking for __attribute__(nonnull)... yes
checking for __attribute__(noreturn)... yes
checking for __attribute__(noreturn_funcptr)... yes
checking for __attribute__(packed)... yes
checking for __attribute__(pure)... yes
checking for __attribute__(sentinel)... yes
checking for __attribute__(unused)... yes
checking for __attribute__(visibility)... yes
checking for __attribute__(warn_unused_result)... yes
checking for __attribute__(weak_alias)... no
checking for __attribute__(destructor)... yes
checking for __attribute__(optnone)... yes
checking for __attribute__(extension)... yes
checking for compiler familyid... 1
checking for compiler familyname... GNU
checking for compiler version... 262657
checking for compiler version_str... 4.2.1

*** Java MPI bindings
checking if want Java bindings... no

*** OpenSHMEM profiling
checking if pshmem will be enabled... no (OpenSHMEM disabled)

*** Assembler
checking dependency style of gcc... gcc3
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking for fgrep... /usr/bin/grep -F
checking for __atomic builtin atomics... yes
checking for __atomic_compare_exchange_n... yes
checking if __atomic_compare_exchange_n() gives correct results... yes
checking if __int128 atomic compare-and-swap is always lock-free... yes
checking if .proc/endp is needed... no
checking directive for setting text section... .text
checking directive for exporting symbols... .globl
checking for objdump... objdump
checking if .note.GNU-stack is needed... /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/objdump: error: 'conftest.o': Invalid/Unsupported object file format
no
checking suffix for labels... :
checking prefix for global symbol labels... none
configure: error: Could not determine global symbol label prefix

% gcc --version
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

% gfortran --version
GNU Fortran (GCC) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

% g++ --version
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

The config log file shows the error

configure:39557: checking prefix for global symbol labels
configure: trying _
configure:39597: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -c conftest.s >conftest.out 2>&1
configure:39600: $? = 0
configure:39605: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -I. conftest_c.c -c > conftest.cmpl 2>&1
configure:39608: $? = 0
configure:39613: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing conftest_c.o conftest.o -o conftest    -lz > conftest.link 2>&1
configure:39616: $? = 1
ld: Undefined symbols:
  _gsym_test_func, referenced from:
      _main in conftest_c.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
configure: failed C program was: 
#ifdef __cplusplus
extern "C" {
#endif
void gsym_test_func(void);
#ifdef __cplusplus
}
#endif
int
main()
{
    gsym_test_func();
    return 0;
}
configure: failed ASM program was: 

.text
# _gsym_test_func
.globl _gsym_test_func
_gsym_test_func:
# _gsym_test_func

configure: trying 
configure:39597: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -c conftest.s >conftest.out 2>&1
configure:39600: $? = 0
configure:39605: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -I. conftest_c.c -c > conftest.cmpl 2>&1
configure:39608: $? = 0
configure:39613: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing conftest_c.o conftest.o -o conftest    -lz > conftest.link 2>&1
configure:39616: $? = 1
ld: Undefined symbols:
  _gsym_test_func, referenced from:
      _main in conftest_c.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
configure: failed C program was: 
#ifdef __cplusplus
extern "C" {
#endif
void gsym_test_func(void);
#ifdef __cplusplus
}
#endif
int
main()
{
    gsym_test_func();
    return 0;
}
configure: failed ASM program was: 

.text
# gsym_test_func
.globl gsym_test_func
gsym_test_func:
# gsym_test_func

configure: trying .
configure:39597: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -c conftest.s >conftest.out 2>&1
configure:39600: $? = 0
configure:39605: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -I. conftest_c.c -c > conftest.cmpl 2>&1
configure:39608: $? = 0
configure:39613: gcc -O3 -DNDEBUG -finline-functions -fno-strict-aliasing conftest_c.o conftest.o -o conftest    -lz > conftest.link 2>&1
configure:39616: $? = 1
ld: Undefined symbols:
  _gsym_test_func, referenced from:
      _main in conftest_c.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
configure: failed C program was: 
#ifdef __cplusplus
extern "C" {
#endif
void gsym_test_func(void);
#ifdef __cplusplus
}
#endif
int
main()
{
    gsym_test_func();
    return 0;
}
configure: failed ASM program was: 

.text
# .gsym_test_func
.globl .gsym_test_func
.gsym_test_func:
# .gsym_test_func

configure:39654: result: none
configure:39658: error: Could not determine global symbol label prefix
jsquyres commented 11 months ago

I'm able to reproduce the issue.

Well, that's awesome (in a rhymes-with-aaaaaaarrrrgggghhh kind of way). We were literally just about to release v4.1.6.

Let us dig into this a little...

jsquyres commented 11 months ago

I note that this issue does not occur on the upcoming v5.0.x branch. We've done a large overhaul of the assembly support in v5.0.x; so it's not entirely unsurprising that the issue is actually fixed over there.

cianciosa commented 11 months ago

Well investigating a different issue in another project someone pointed out a there's a new linker.

I can confirm that by using the command

./configure LDFLAGS="-ld_classic"

invokes the old linker which doesn't have this issue.

fxcoudert commented 11 months ago

(Hi, Homebrew maintainer here.) Yes, the new linker has quite a number of missing features or bugs. But the old linker will go away at some point, so I would encourage all affected users to file a Feedback with Apple (and share the number here, please). Any regression should be treated: what is the ASM that was working before, and how does it behave now.

The workaround of the classic linker is a good way to build for now, but it is only a workaround.

fxcoudert commented 11 months ago

Reported to Apple as FB13194320. Content of the report is copied there: https://gist.github.com/fxcoudert/7127b8f92223dd2131f191b8b5f6e549

bwbarrett commented 11 months ago

We'll have a patch to fix out today. We had actually removed the hand-coded assembly files before the 4.0 series started, but as an oversight left some of the configuration tests in place. We're going to remove the (unneeded and now broken) tests and the Apple linker will be happy.

bosilca commented 11 months ago

I don't think the issue is the linker. Empty assembly functions end up not being in the compiled file, and this leads to inability to link against. There is a really simple fix:

diff --git a/config/opal_config_asm.m4 b/config/opal_config_asm.m4
index 5183c7e082..282ac5dc04 100644
--- a/config/opal_config_asm.m4
+++ b/config/opal_config_asm.m4
@@ -474,6 +474,7 @@ AC_DEFUN([OPAL_CHECK_ASM_PROC],[
                     OPAL_TRY_ASSEMBLE([
      .proc mysym
 mysym:
+         ret
      .endp mysym],
                           [opal_cv_asm_need_proc="yes"])
                     rm -f conftest.out])

It can be augmented with the next patch to prevent picky compilers from complaining about incorrect prototypes.

diff --git a/config/opal_config_asm.m4 b/config/opal_config_asm.m4
index 5183c7e082..282ac5dc04 100644
--- a/config/opal_config_asm.m4
+++ b/config/opal_config_asm.m4
@@ -527,7 +528,7 @@ void gsym_test_func(void);
 }
 #endif
 int
-main()
+main(void)
 {
     gsym_test_func();
     return 0;

Note, that these patches need to be propagated upstream to all projects inheriting our asm detection.

jsquyres commented 11 months ago

11942 has been merged to the v4.1.x branch, which should address this issue.

@cianciosa @fxcoudert Could you try building the HEAD of the v4.1.x branch? Or you can wait for a new snapshot tarball to be generated this evening (https://www.open-mpi.org/nightly/v4.1.x/).

cianciosa commented 11 months ago

@jsquyres I can confirm there are no more configuration errors.

jsquyres commented 11 months ago

I'm going to close this as fixed; @fxcoudert let us know if you run any further issues.