signbit issue for macOS 10.14 and 10.15 on Homebrew

henryiii commented 3 years ago

[x] Checked for duplicates

Should build? :)

To Reproduce

I don't have a 10.14 or 10.15 machine to test on, if I did, this should do it:

cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core
gh pr checkout 75166
brew install --build-from-source root

Setup

ROOT 6.24.00, macOS < 11, C++17 mode.

Additional context

Full error:

While building module 'Core':
While building module 'std' imported from input_line_1:1:
In file included from <module-includes>:31:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/complex.h:28:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/ccomplex:20:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/complex:245:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:317:9: error: missing '#include "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h"'; declaration of 'signbit' must be imported from module 'std.depr.math_h' before it is required
using ::signbit;
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h:331:1: note: previous declaration is here
signbit(_A1 __lcpp_x) _NOEXCEPT
^
input_line_1:1:10: fatal error: could not build module 'std'
#include <new>
 ~~~~~~~~^
Warning in cling::IncrementalParser::CheckABICompatibility():
  Failed to extract C++ standard library version.
Warning in cling::IncrementalParser::CheckABICompatibility():
  Possible C++ standard library mismatch, compiled with _LIBCPP_ABI_VERSION '1'
  Extraction of runtime standard library version was: ''
While building module 'Core':
While building module 'Cling_Runtime' imported from input_line_2:1:
In file included from <module-includes>:1:
/private/tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/etc/cling/Interpreter/RuntimeUniverse.h:27:10: fatal error: could not build module 'std'
#include <new>
 ~~~~~~~~^
/private/tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/etc/cling/Interpreter/DynamicExprInfo.h:13:10: fatal error: could not build module 'std'
#include <string>
 ~~~~~~~~^
While building module 'Core':
While building module 'ROOT_Rtypes' imported from /tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/include/Rtypes.h:23:
In file included from <module-includes>:1:
/tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/include/RtypesCore.h:28:10: fatal error: could not build module 'std'
#include <cstddef> // size_t, NULL
 ~~~~~~~~^
In file included from input_line_3:2:
/tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/include/Rtypes.h:23:10: fatal error: could not build module 'ROOT_Rtypes'
#include "RtypesCore.h"
 ~~~~~~~~^~~~~~~~~~~~~~
While building module 'Core':
While building module 'ROOT_Foundation_C' imported from /tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/include/Rtypes.h:29:
In file included from <module-includes>:1:
/tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/include/ThreadLocalStorage.h:61:10: fatal error: could not build module 'ROOT_Rtypes'
#include "RtypesCore.h"
 ~~~~~~~~^~~~~~~~~~~~~~
While building module 'Core':
While building module 'ROOT_Foundation_Stage1_NoRTTI' imported from /tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/include/Rtypes.h:195:
In file included from <module-includes>:2:
/tmp/root-20210414-13675-19zg57u/root-6.24.00/builddir/include/ROOT/RStringView.hxx:19:10: fatal error: could not build module 'std'
#include <string_view>
 ~~~~~~~~^
Error: Error loading the default rootcling header files.
[7241/9610] Building CXX object core/metacling/src/CMakeFiles/MetaCling.dir/TCling.cxx.o
ninja: build stopped: subcommand failed.

Note there have been signbit issues before, https://github.com/Homebrew/homebrew-core/pull/75166/files#r613512702

henryiii commented 3 years ago

Anyone have ideas? I've removed the old workaround line, seems to be the same problem. Works just fine on macOS 11 (Intel and ARM), but breaks on 10.14 and 10.15. Is there a way to change the build procedure to an older (non-modules?) one? Could be done just on 10.14 and 10.15.

henryiii commented 3 years ago

Turning off runtime cxx_modules for 10.15 and 10.14 causes the cling step to still fail, in the Generating G__Core.cxx, ../lib/libCore.rootmap step instead of the module generation. Same sorts of errors, though.

henryiii commented 3 years ago

@chrisburr have you seen anything like this when updating ROOT? Didn't see anything going into the conda-forge formula related, but might have missed something.

chrisburr commented 3 years ago

We don't see any issues in conda-forge and successfully target 10.14+. It's using the conda-forge provided compilers (Clang 11) though so I could imagine this is different.

chrisburr commented 3 years ago

Actually there was an issue reported when getting expat from the macOS SDK: https://mattermost.web.cern.ch/root/pl/zdaib63hnbrctqk5sw43gy1t8r

Maybe you need to get some more dependencies from brew.

vgvassilev commented 3 years ago

@henryiii, the gh pr checkout 75166 command does not work for me.

Axel-Naumann commented 3 years ago

Just reformatting the relevant part as that's off screen for me:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:317:9: 
error: missing '#include "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/math.h"';
declaration of 'signbit' must be imported from module 'std.depr.math_h' before it is required

With Xcode 12.5 I have in /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/cmath:308

#include <math.h>

Does the build node have that, too? Which Xcode version is this? (Mine has using ::signbit; in line 321.)

henryiii commented 3 years ago

@Axel-Naumann 10.14 has 11.3.1, 10.15 has 12.4, and 11 has 12.4 as well.

@vgvassilev Just get bump-root-6.24.00 from git@github.com:chenrui333/homebrew-core.git somehow, I would use the GitHub command line app gh, but you can use vanilla git instead if you wish.

I've pushed a build that should have useful logs; 10.14 I'm building with runtime modules off, 10.15+ with runtime modules on. So we'll have one of each possible outcome.

Axel-Naumann commented 3 years ago

Thanks. I see line cmath:304 has an unprotected, always visible #include <math.h> so it looks like an issue with C++ modules. I'll leave that for you, @vgvassilev :-)

henryiii commented 3 years ago

@vgvassilev have you looked into that include?

vgvassilev commented 3 years ago

@henryiii, I get:

cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core
git remote add root git@github.com:chenrui333/homebrew-core.git
git fetch root
git checkout  bump-root-6.24.00
brew install --build-from-source root
==> Downloading https://homebrew.bintray.com/bottles/cmake-3.20.2.catalina.bottle.tar.gz
#=#=#                                                                         
curl: (22) The requested URL returned error: 403 Forbidden
Error: Failed to download resource "cmake"

henryiii commented 3 years ago

You need to brew update first, bintray shut down May 1st and brew had to migrate storage providers. You need brew 3.1+ to download packages.

vgvassilev commented 3 years ago

I have built successfully outside of brew. While building the G__Core.cxx these are the differences in the rootcling setup -- https://www.diffchecker.com/2v8JFoGU, https://www.diffchecker.com/svLt7sk1

Nothing really major, only the broken version prepends /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include before the path to libc++. The broken version also specifies the concrete sdk version (MacOSX10.15 as opposed to just MacOSX) but that does not seem major...

henryiii commented 3 years ago

This didn't/doesn't break ROOT 6.22, only 6.24, though. Any thoughts on how to proceed?

vgvassilev commented 3 years ago

Looks like we need SDKROOT workaround used for 6.22 back: git diff bump-root-6.22.06 Formula/root.rb

   ...
   def install
-    # Work around "error: no member named 'signbit' in the global namespace"
-    ENV.delete("SDKROOT") if DevelopmentTools.clang_build_version >= 900
-
     # Freetype/afterimage/gl2ps/lz4 are vendored in the tarball, so are fine.
     # However, this is still permitting the build process to make remote
     # connections. As a hack, since upstream support it, we inreplace
@@ -96,6 +81,10 @@ class Root < Formula
     cxx_version = (MacOS.version < :mojave) ? 14 : 17
     args << "-DCMAKE_CXX_STANDARD=#{cxx_version}"

+    # TODO: probably can be removed
+    runtime_cxxmodules = (MacOS.version >= :catalina) ? "ON" : "OFF"
+    args << "-Druntime_cxxmodules=#{runtime_cxxmodules}"
+
     # Homebrew now sets CMAKE_INSTALL_LIBDIR to /lib, which is incorrect
     # for ROOT with gnuinstall, so we set it back here.
     args << "-DCMAKE_INSTALL_LIBDIR=lib/root"

henryiii commented 3 years ago

That workaround no longer works from what I understand, it's now a no-op. But it's also not needed, 6.22 builds just fine without it. I can give it a try if you'd like me to.

henryiii commented 3 years ago

Actually, the first few times this built, the workaround was still there. So it has been tested with this line.

Yeah I don't think we even allow the SDKROOT env to be set in the superenv anymore

vgvassilev commented 3 years ago

Okay, so it seems to be some setup issue as the signbit error was in v6.22. I would recommend to build v6.22 and v6.24 and compare the CMake files in the builddir.

It may be that, for some reason, homebrew setup makes rootcling prepend /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include.

PS: I will put my money on the -[i]sysroot compiler flags ;)

henryiii commented 3 years ago

Hmm, would this line perhaps be a problem?

args << "-DCMAKE_OSX_SYSROOT=#{MacOS.sdk_for_formula(self).path}" if MacOS.sdk_root_needed?

That's when making std_cmake_args. (Trying a debugging run)

henryiii commented 3 years ago

Well, on each version:

-DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk # macOS 11 build
-DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk # macOS 10.15 build
-DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk # macOS 10.14 build

If you manually remove this line, they all break with a signbit issue.

vgvassilev commented 3 years ago

Ok, some progress. I can reproduce this outside of homebrew with the following configuration:

 cat recmake_initial.sh 
#!/bin/sh
rm -f CMakeCache.txt
/usr/local/Cellar/cmake/3.20.2/bin/cmake  -DBUILD_TESTING="OFF" -DCLING_CXX_PATH="clang++" -DCMAKE_BUILD_TYPE="Release" -DCMAKE_CXX_FLAGS_RELEASE="-DNDEBUG" -DCMAKE_CXX_STANDARD="17" -DCMAKE_C_FLAGS_RELEASE="-DNDEBUG" -DCMAKE_FIND_FRAMEWORK="LAST" -DCMAKE_INSTALL_ELISPDIR="/usr/local/Cellar/root/6.24.00/share/emacs/site-lisp/root" -DCMAKE_INSTALL_LIBDIR="lib/root" -DCMAKE_INSTALL_PREFIX="/usr/local/Cellar/root/6.24.00" -DCMAKE_OSX_SYSROOT="/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk" -DCMAKE_VERBOSE_MAKEFILE="ON" -DPYTHON_EXECUTABLE="/usr/local/opt/python@3.9/bin/python3" -Dbuiltin_cfitsio="OFF" -Dbuiltin_freetype="ON" -Dbuiltin_glew="ON" -Ddavix="ON" -Dfftw3="ON" -Dfitsio="ON" -Dfortran="ON" -Dgdml="ON" -Dgnuinstall="ON" -Dimt="ON" -Dmathmore="ON" -Dminuit2="ON" -Dmysql="OFF" -Dpgsql="OFF" -Dpyroot="ON" -Droofit="ON" -Druntime_cxxmodules="ON" -Dssl="ON" -Dtmva="ON" -Dxrootd="ON" /Users/vvassilev/workspace/sources/root -G Ninja

First couple of remarks -- we seem to have options that we do not need. Eg CMAKE_VERBOSE_MAKEFILE (we build with Ninja), CMAKE_C_FLAGS_RELEASE (should leave ROOT to decide what's best), etc...

The failing command is:

(cd /Users/vvassilev/workspace/builds/scratch/root_homebrew/core && /usr/local/Cellar/cmake/3.20.2/bin/cmake -E env LD_LIBRARY_PATH=/Users/vvassilev/workspace/builds/scratch/root_homebrew/lib: /Users/vvassilev/workspace/builds/scratch/root_homebrew/core/rootcling_stage1/src/rootcling_stage1 -v2 -f G__Core.cxx -cxxmodule -s /Users/vvassilev/workspace/builds/scratch/root_homebrew/lib/libCore.so -excludePath /Users/vvassilev/workspace/sources/root -excludePath /Users/vvassilev/workspace/builds/scratch/root_homebrew/ginclude -excludePath /Users/vvassilev/workspace/builds/scratch/root_homebrew/externals -excludePath /Users/vvassilev/workspace/builds/scratch/root_homebrew/builtins -writeEmptyRootPCM -m _Builtin_intrinsics -mByproduct _Builtin_intrinsics -mByproduct ROOT_Foundation_Stage1_NoRTTI -mByproduct ROOT_Foundation_C -mByproduct ROOT_Rtypes -D__STDC_NO_COMPLEX__ -D__COMPLEX_H__ -D_COMPLEX_H -I/Users/vvassilev/workspace/builds/scratch/root_homebrew/include -I/Users/vvassilev/workspace/builds/scratch/root_homebrew/ginclude -I/Users/vvassilev/workspace/sources/root/core/base/inc -I/Users/vvassilev/workspace/sources/root/core/foundation/inc -I/Users/vvassilev/workspace/sources/root/core/cont/inc -I/Users/vvassilev/workspace/sources/root/core/gui/inc -I/Users/vvassilev/workspace/sources/root/core/meta/inc -I/Users/vvassilev/workspace/sources/root/core/clib/inc -I/Users/vvassilev/workspace/sources/root/core/rint/inc -I/Users/vvassilev/workspace/sources/root/core/zip/inc -I/Users/vvassilev/workspace/sources/root/core/thread/inc -I/Users/vvassilev/workspace/sources/root/core/textinput/inc -I/Users/vvassilev/workspace/sources/root/core/clingutils/inc -I/Users/vvassilev/workspace/sources/root/core/base/v7/inc -I/Users/vvassilev/workspace/sources/root/core/foundation/v7/inc -I/Users/vvassilev/workspace/sources/root/core/unix/inc -I/Users/vvassilev/workspace/sources/root/core/macosx/inc -I/usr/local/include -I/Users/vvassilev/workspace/sources/root/builtins/xxhash -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include ROOT/TErrorDefaultHandler.hxx ROOT/TSequentialExecutor.hxx ROOT/StringConv.hxx Buttons.h Bytes.h Byteswap.h KeySymbols.h MessageTypes.h Riostream.h Rtypes.h TApplication.h TAtt3D.h TAttAxis.h TAttBBox2D.h TAttBBox.h TAttFill.h TAttLine.h TAttMarker.h TAttPad.h TAttText.h TBase64.h TBenchmark.h TBuffer3D.h TBuffer3DTypes.h TBuffer.h TColor.h TColorGradient.h TDatime.h TDirectory.h TEnv.h TException.h TExec.h TFileCollection.h TFileInfo.h TFolder.h TInetAddress.h TMacro.h TMathBase.h TMD5.h TMemberInspector.h TMessageHandler.h TNamed.h TNotifyLink.h TObject.h TObjString.h TParameter.h TPluginManager.h TPoint.h TPRegexp.h TProcessID.h TProcessUUID.h TQClass.h TQCommand.h TQConnection.h TQObject.h TRedirectOutputGuard.h TRefCnt.h TRef.h TRegexp.h TRemoteObject.h TROOT.h TRootIOCtor.h TStopwatch.h TStorage.h TString.h TStringLong.h TStyle.h TSysEvtHandler.h TSystemDirectory.h TSystemFile.h TSystem.h TTask.h TThreadSlots.h TTime.h TTimer.h TTimeStamp.h TUri.h TUrl.h TUUID.h TVersionCheck.h TVirtualAuth.h TVirtualFFT.h TVirtualGL.h TVirtualMonitoring.h TVirtualMutex.h TVirtualPadEditor.h TVirtualPad.h TVirtualPadPainter.h TVirtualPerfStats.h TVirtualPS.h TVirtualQConnection.h TVirtualRWMutex.h TVirtualTableInterface.h TVirtualViewer3D.h TVirtualX.h ROOT/RDirectoryEntry.hxx ROOT/RIndexIter.hxx strlcpy.h snprintf.h strtok.h ROOT/TSeq.hxx TArrayC.h TArrayD.h TArrayF.h TArray.h TArrayI.h TArrayL64.h TArrayL.h TArrayS.h TBits.h TBtree.h TClassTable.h TClonesArray.h TCollection.h TCollectionProxyInfo.h TExMap.h THashList.h THashTable.h TIterator.h TList.h TMap.h TObjArray.h TObjectTable.h TOrdCollection.h TRefArray.h TRefTable.h TSeqCollection.h TSortedList.h TVirtualCollectionProxy.h ESTLType.h RStringView.h TClassEdit.h TError.h ROOT/RIntegerSequence.hxx ROOT/RLogger.hxx ROOT/RMakeUnique.hxx ROOT/RNotFn.hxx ROOT/RSpan.hxx ROOT/RStringView.hxx ROOT/span.hxx ROOT/TypeTraits.hxx TMacOSXSystem.h TUnixSystem.h root_std_complex.h GuiTypes.h TApplicationImp.h TBrowser.h TBrowserImp.h TCanvasImp.h TClassMenuItem.h TContextMenu.h TContextMenuImp.h TControlBarImp.h TGuiFactory.h TInspectorImp.h TObjectSpy.h TToggleGroup.h TToggle.h TBaseClass.h TClassGenerator.h TClass.h TClassRef.h TClassStreamer.h TDataMember.h TDataType.h TDictAttributeMap.h TDictionary.h TEnumConstant.h TEnum.h TFileMergeInfo.h TFunction.h TFunctionTemplate.h TGenericClassInfo.h TGlobal.h TInterpreter.h TInterpreterValue.h TIsAProxy.h TListOfDataMembers.h TListOfEnums.h TListOfEnumsWithLock.h TListOfFunctions.h TListOfFunctionTemplates.h TMemberStreamer.h TMethodArg.h TMethodCall.h TMethod.h TProtoClass.h TRealData.h TSchemaHelper.h TSchemaRule.h TSchemaRuleSet.h TStatusBitsChecker.h TStreamerElement.h TStreamer.h TVirtualIsAProxy.h TVirtualRefProxy.h TVirtualStreamerInfo.h TVirtualArray.h TVirtualObject.h Getline.h /Users/vvassilev/workspace/sources/root/core/base/inc/LinkDef.h)

For some reason, homebrew (or some of the flags from the cmake configuration above) decides to add: -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include which makes the command fail.

This cmake configuration fixes the issue for me: cmake -DBUILD_TESTING="OFF" -DCLING_CXX_PATH="clang++" -DCMAKE_BUILD_TYPE="Release" -DCMAKE_CXX_FLAGS_RELEASE="-DNDEBUG" -DCMAKE_CXX_STANDARD="17" -DCMAKE_INSTALL_ELISPDIR="/usr/local/Cellar/root/6.24.00/share/emacs/site-lisp/root" -DCMAKE_INSTALL_LIBDIR="lib/root" -DCMAKE_INSTALL_PREFIX="/usr/local/Cellar/root/6.24.00" -DPYTHON_EXECUTABLE="/usr/local/opt/python@3.9/bin/python3" -Dbuiltin_cfitsio="OFF" -Dbuiltin_freetype="ON" -Dbuiltin_glew="ON" -Ddavix="ON" -Dfftw3="ON" -Dfitsio="ON" -Dfortran="ON" -Dgdml="ON" -Dgnuinstall="ON" -Dimt="ON" -Dmathmore="ON" -Dminuit2="ON" -Dmysql="OFF" -Dpgsql="OFF" -Dpyroot="ON" -Droofit="ON" -Druntime_cxxmodules="ON" -Dssl="ON" -Dtmva="ON" -Dxrootd="ON" /Users/vvassilev/workspace/sources/root -G Ninja

henryiii commented 3 years ago

CMAKE_C_FLAGS_RELEASE (should leave ROOT to decide what's best

Homebrew has specific flags and such that are supposed to always be used on all projects. That's where these come from, a list called "cmake_std_args" in homebrew. Meddling with these is very tricky - but we only need it for older versions of macOS, 11 builds fine, so I think it will likely make it through review. We can try, anyway. I'll have to manually filter out settings from that list.

vgvassilev commented 3 years ago

@henryiii, can we mark this issue as resolved?

henryiii commented 3 years ago

Should be cleared up. Using depends_on :xcode if MacOS.version <= :catalina, this gets the right paths and works. Will close here, and hopefully the PR will be merged soon in homebrew.

henryiii commented 3 years ago

Thanks, @vgvassilev!!! Once the problem was identified as the issue with those paths, a very nice and simple solution became obvious.

vgvassilev commented 3 years ago

No worries, thanks for pulling this off!

github-actions[bot] commented 3 years ago

Hi @henryiii, @vgvassilev,

It appears this issue is closed, but wasn't yet added to a project. Please add upcoming versions that will include the fix, or 'not applicable' otherwise.

Sincerely, :robot:

root-project / root