Bundled libcpuid fails to build on macOS-10.14 VM image

[lohedges]

As of March 23rd, we will need to upgrade the macOS VM image used by our Azure Pipeline to version 10.14 since this will be the oldest image that is available. I've tried doing this already, but currently the build fails. (See here for an example.)

It looks like the bundled libcpuid fails to compile under 10.14. I've tested reverting the image and the build works fine. Strangely I have no issues compiling Sire on a MacBook running an even newer macOS.

[lohedges]

Actually, having updated and restarted the MacBook it now no longer compiles there either. Output from running ./compile_sire.sh:

checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking for gcc... /Users/lester/sire.app/bin/clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... configure: error: in `/Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
-- Patience... Compiling libcpuid...
make: *** No targets specified and no makefile found.  Stop.
-- Patience... Installing libcpuid...
make: *** No rule to make target 'install'.  Stop.
error: /Library/Developer/CommandLineTools/usr/bin/install_name_tool: can't open file: /Users/lester/sire.app/pkgs/sire-2019.3.0/bundled/lib/libcpuid.dylib (No such file or directory)
CMake Error at src/bundled/install_cpuid.cmake:127 (message):
  Cannot find CPUID library
  /Users/lester/sire.app/pkgs/sire-2019.3.0/bundled/lib/libcpuid.dylib.
  Error with compile?
Call Stack (most recent call first):
  CMakeLists.txt:765 (include)

-- Configuring incomplete, errors occurred!
See also "/Users/lester/Code/Sire/build/corelib/CMakeFiles/CMakeOutput.log".
See also "/Users/lester/Code/Sire/build/corelib/CMakeFiles/CMakeError.log".
SOMETHING WENT WRONG WHEN USING CMAKE ON CORELIB!

This full contents of config.log can be found below:

Click to expand

``` This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. It was created by libcpuid CPU Identification library configure 0.4.1, which was generated by GNU Autoconf 2.69. Invocation command line was $ /Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/configure --enable-static=no --enable-shared=yes --prefix=/Users/lester/sire.app/pkgs/sire-2019.3.0/bundled --libdir=/Users/lester/sire.app/pkgs/sire-2019.3.0/bundled/lib CC=/Users/lester/sire.app/bin/clang CFLAGS=-DHAVE_STDINT_H ## --------- ## ## Platform. ## ## --------- ## hostname = roll.local uname -m = x86_64 uname -r = 18.7.0 uname -s = Darwin uname -v = Darwin Kernel Version 18.7.0: Thu Jan 23 06:52:12 PST 2020; root:xnu-4903.278.25~1/RELEASE_X86_64 /usr/bin/uname -p = i386 /bin/uname -X = unknown /bin/arch = unknown /usr/bin/arch -k = unknown /usr/convex/getsysinfo = unknown /usr/bin/hostinfo = Mach kernel version: Darwin Kernel Version 18.7.0: Thu Jan 23 06:52:12 PST 2020; root:xnu-4903.278.25~1/RELEASE_X86_64 Kernel configured for up to 4 processors. 2 processors are physically available. 4 processors are logically available. Processor type: x86_64h (Intel x86-64h Haswell) Processors active: 0 1 2 3 Primary memory available: 16.00 gigabytes Default processor set: 350 tasks, 1362 threads, 4 processors Load average: 3.18, Mach factor: 1.10 /bin/machine = unknown /usr/bin/oslevel = unknown /bin/universe = unknown PATH: /Users/lester/sire.app/bin PATH: /Users/lester/sire.app/condabin PATH: /usr/local/gromacs/bin PATH: /usr/local/bin PATH: /usr/local/sbin PATH: /usr/bin PATH: /bin PATH: /usr/sbin PATH: /sbin PATH: /usr/local/MacGPG2/bin PATH: /usr/local/munki PATH: /opt/X11/bin ## ----------- ## ## Core tests. ## ## ----------- ## configure:2272: checking for a BSD-compatible install configure:2340: result: /usr/local/bin/ginstall -c configure:2351: checking whether build environment is sane configure:2406: result: yes configure:2557: checking for a thread-safe mkdir -p configure:2596: result: /usr/local/bin/gmkdir -p configure:2603: checking for gawk configure:2633: result: no configure:2603: checking for mawk configure:2633: result: no configure:2603: checking for nawk configure:2633: result: no configure:2603: checking for awk configure:2619: found /usr/bin/awk configure:2630: result: awk configure:2641: checking whether make sets $(MAKE) configure:2663: result: yes configure:2692: checking whether make supports nested variables configure:2709: result: yes configure:2846: checking whether make supports nested variables configure:2863: result: yes configure:2935: checking for gcc configure:2962: result: /Users/lester/sire.app/bin/clang configure:3191: checking for C compiler version configure:3200: /Users/lester/sire.app/bin/clang --version >&5 clang version 4.0.1 (tags/RELEASE_401/final) Target: x86_64-apple-darwin18.7.0 Thread model: posix InstalledDir: /Users/lester/sire.app/bin configure:3211: $? = 0 configure:3200: /Users/lester/sire.app/bin/clang -v >&5 clang version 4.0.1 (tags/RELEASE_401/final) Target: x86_64-apple-darwin18.7.0 Thread model: posix InstalledDir: /Users/lester/sire.app/bin configure:3211: $? = 0 configure:3200: /Users/lester/sire.app/bin/clang -V >&5 clang-4.0: error: argument to '-V' is missing (expected 1 value) clang-4.0: error: no input files configure:3211: $? = 1 configure:3200: /Users/lester/sire.app/bin/clang -qversion >&5 clang-4.0: error: unknown argument: '-qversion' clang-4.0: error: no input files configure:3211: $? = 1 configure:3231: checking whether the C compiler works configure:3253: /Users/lester/sire.app/bin/clang -DHAVE_STDINT_H conftest.c >&5 configure:3257: $? = 0 configure:3305: result: yes configure:3308: checking for C compiler default output file name configure:3310: result: a.out configure:3316: checking for suffix of executables configure:3323: /Users/lester/sire.app/bin/clang -o conftest -DHAVE_STDINT_H conftest.c >&5 configure:3327: $? = 0 configure:3349: result: configure:3371: checking whether we are cross compiling configure:3379: /Users/lester/sire.app/bin/clang -o conftest -DHAVE_STDINT_H conftest.c >&5 conftest.c:11:10: fatal error: 'stdio.h' file not found #include ^~~~~~~~~ 1 error generated. configure:3383: $? = 1 configure:3390: ./conftest /Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/configure: line 3392: ./conftest: No such file or directory configure:3394: $? = 127 configure:3401: error: in `/Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1': configure:3403: error: cannot run C compiled programs. If you meant to cross compile, use `--host'. See `config.log' for more details ## ---------------- ## ## Cache variables. ## ## ---------------- ## ac_cv_env_CC_set=set ac_cv_env_CC_value=/Users/lester/sire.app/bin/clang ac_cv_env_CFLAGS_set=set ac_cv_env_CFLAGS_value=-DHAVE_STDINT_H ac_cv_env_CPPFLAGS_set= ac_cv_env_CPPFLAGS_value= ac_cv_env_CPP_set= ac_cv_env_CPP_value= ac_cv_env_LDFLAGS_set= ac_cv_env_LDFLAGS_value= ac_cv_env_LIBS_set= ac_cv_env_LIBS_value= ac_cv_env_LT_SYS_LIBRARY_PATH_set= ac_cv_env_LT_SYS_LIBRARY_PATH_value= ac_cv_env_build_alias_set= ac_cv_env_build_alias_value= ac_cv_env_host_alias_set= ac_cv_env_host_alias_value= ac_cv_env_target_alias_set= ac_cv_env_target_alias_value= ac_cv_path_install='/usr/local/bin/ginstall -c' ac_cv_path_mkdir=/usr/local/bin/gmkdir ac_cv_prog_AWK=awk ac_cv_prog_ac_ct_CC=/Users/lester/sire.app/bin/clang ac_cv_prog_make_make_set=yes am_cv_make_support_nested_variables=yes ## ----------------- ## ## Output variables. ## ## ----------------- ## ACLOCAL='${SHELL} /Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/missing aclocal-1.16' AMDEPBACKSLASH='' AMDEP_FALSE='' AMDEP_TRUE='' AMTAR='$${TAR-tar}' AM_BACKSLASH='\' AM_CPPFLAGS='' AM_DEFAULT_V='$(AM_DEFAULT_VERBOSITY)' AM_DEFAULT_VERBOSITY='0' AM_LDFLAGS='' AM_V='$(V)' AR='' AUTOCONF='${SHELL} /Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/missing autoconf' AUTOHEADER='${SHELL} /Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/missing autoheader' AUTOMAKE='${SHELL} /Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/missing automake-1.16' AWK='awk' CC='/Users/lester/sire.app/bin/clang' CCDEPMODE='' CFLAGS='-DHAVE_STDINT_H' CPP='' CPPFLAGS='' CYGPATH_W='echo' DEFS='' DEPDIR='' DLLTOOL='' DOXYGEN='' DSYMUTIL='' DUMPBIN='' ECHO_C='\c' ECHO_N='' ECHO_T='' EGREP='' EXEEXT='' FGREP='' GREP='' HAVE_DOXYGEN_FALSE='' HAVE_DOXYGEN_TRUE='' INSTALL_DATA='${INSTALL} -m 644' INSTALL_PROGRAM='${INSTALL}' INSTALL_SCRIPT='${INSTALL}' INSTALL_STRIP_PROGRAM='$(install_sh) -c -s' LD='' LDFLAGS='' LIBCPUID_AGE='0' LIBCPUID_CURRENT='14' LIBCPUID_CURRENT_MIN='14' LIBCPUID_REVISION='1' LIBCPUID_VERSION_INFO='14:1:0' LIBOBJS='' LIBS='' LIBTOOL='' LIPO='' LN_S='' LTLIBOBJS='' LT_SYS_LIBRARY_PATH='' MAKEINFO='${SHELL} /Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/missing makeinfo' MANIFEST_TOOL='' MKDIR_P='/usr/local/bin/gmkdir -p' NM='' NMEDIT='' OBJDUMP='' OBJEXT='' OTOOL64='' OTOOL='' PACKAGE='libcpuid' PACKAGE_BUGREPORT='libcpuid-devel@lists.sourceforge.net' PACKAGE_NAME='libcpuid CPU Identification library' PACKAGE_STRING='libcpuid CPU Identification library 0.4.1' PACKAGE_TARNAME='libcpuid' PACKAGE_URL='' PACKAGE_VERSION='0.4.1' PATH_SEPARATOR=':' RANLIB='' SED='' SET_MAKE='' SHELL='/bin/sh' STRIP='' VERSION='0.4.1' ac_ct_AR='' ac_ct_CC='/Users/lester/sire.app/bin/clang' ac_ct_DUMPBIN='' am__EXEEXT_FALSE='' am__EXEEXT_TRUE='' am__fastdepCC_FALSE='' am__fastdepCC_TRUE='' am__include='' am__isrc='' am__leading_dot='.' am__nodep='' am__quote='' am__tar='$${TAR-tar} chof - "$$tardir"' am__untar='$${TAR-tar} xf -' bindir='${exec_prefix}/bin' build='' build_alias='' build_cpu='' build_os='' build_vendor='' datadir='${datarootdir}' datarootdir='${prefix}/share' docdir='${datarootdir}/doc/${PACKAGE_TARNAME}' dvidir='${docdir}' exec_prefix='NONE' host='' host_alias='' host_cpu='' host_os='' host_vendor='' htmldir='${docdir}' includedir='${prefix}/include' infodir='${datarootdir}/info' install_sh='${SHELL} /Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/install-sh' libdir='/Users/lester/sire.app/pkgs/sire-2019.3.0/bundled/lib' libexecdir='${exec_prefix}/libexec' localedir='${datarootdir}/locale' localstatedir='${prefix}/var' mandir='${datarootdir}/man' mkdir_p='$(MKDIR_P)' oldincludedir='/usr/include' pdfdir='${docdir}' prefix='/Users/lester/sire.app/pkgs/sire-2019.3.0/bundled' program_transform_name='s,x,x,' psdir='${docdir}' sbindir='${exec_prefix}/sbin' sharedstatedir='${prefix}/com' sysconfdir='${prefix}/etc' target_alias='' ## ----------- ## ## confdefs.h. ## ## ----------- ## /* confdefs.h */ #define PACKAGE_NAME "libcpuid CPU Identification library" #define PACKAGE_TARNAME "libcpuid" #define PACKAGE_VERSION "0.4.1" #define PACKAGE_STRING "libcpuid CPU Identification library 0.4.1" #define PACKAGE_BUGREPORT "libcpuid-devel@lists.sourceforge.net" #define PACKAGE_URL "" #define PACKAGE "libcpuid" #define VERSION "0.4.1" configure: exit 1 ```

This section contains some clang related errors:

configure:3200: /Users/lester/sire.app/bin/clang -V >&5
clang-4.0: error: argument to '-V' is missing (expected 1 value)
clang-4.0: error: no input files
configure:3211: $? = 1
configure:3200: /Users/lester/sire.app/bin/clang -qversion >&5
clang-4.0: error: unknown argument: '-qversion'
clang-4.0: error: no input files
configure:3211: $? = 1
configure:3231: checking whether the C compiler works
configure:3253: /Users/lester/sire.app/bin/clang -DHAVE_STDINT_H   conftest.c  >&5
configure:3257: $? = 0
configure:3305: result: yes
configure:3308: checking for C compiler default output file name
configure:3310: result: a.out
configure:3316: checking for suffix of executables
configure:3323: /Users/lester/sire.app/bin/clang -o conftest -DHAVE_STDINT_H   conftest.c  >&5
configure:3327: $? = 0
configure:3349: result: 
configure:3371: checking whether we are cross compiling
configure:3379: /Users/lester/sire.app/bin/clang -o conftest -DHAVE_STDINT_H   conftest.c  >&5
conftest.c:11:10: fatal error: 'stdio.h' file not found
#include <stdio.h>
         ^~~~~~~~~
1 error generated.
configure:3383: $? = 1
configure:3390: ./conftest
/Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1/configure: line 3392: ./conftest: No such file or directory
configure:3394: $? = 127
configure:3401: error: in `/Users/lester/Code/Sire/build/corelib/build_bundled/libcpuid-0.4.1':
configure:3403: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

[lohedges]

Moving to the build/corelib/build_bundled/libcpuid-0.4.1 directory and manually running ./configure followed by make works just fine, so I assume that there's some issue with the way the build is configured in corelib/src/bundled/install_cpuid.cmake.

[lohedges]

This is one of those annoying issues where simply re-running ./compile_sire.sh allows the build to proceed without any further issue!

Having searched around I thought that the problem was likely a result of Apple moving the XCode development headers out of /usr/include as of macOS 10.14. This can be fixed from the command line (albeit requiring sudo access) as described here, so we might be able to do this on the macOS VM image prior to building Sire. I had hoped that installing the one of the additional clangdev or clang-tools packages from conda-forge may have fixed the problem, but no luck. (This also bumps the clang version from 4 to 9, so we might not want to do this anyway.)

I wonder whether the first time round it tries to use the system headers, which don't exist in a standard path so fail, then the second time round it finds the ones installed by conda? That said, I would have thought they should have been visible at the first attempt at compiling.

[lohedges]

I can confirm that installing the XCode command-line tools, as per the instructons here, fixed the problem. Annoyingly the command-line variant of the install failed for me, so I had to manually download the headers and run the installer. Not sure if the same problem would occur on the VM image. (I think I had an old install of the command-line tools that was causing issues.)

[lohedges]

Adding the following line before compiling Sire in the macOS pipeline did the trick:

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

Hopefully that's it with the Sire macOS issues for now. Now to fix the BIoSimSpace macOS Conda corruption issue. I'm keeping my fingers crossed that the updated 10.14 build will sort that...