search

visit-dav / visit

VisIt - Visualization and Data Analysis for Mesh-based Scientific Data
https://visit.llnl.gov
BSD 3-Clause "New" or "Revised" License
437 stars 116 forks source link

3.4.1RC `build_visit --qt6` fails on macOS if Homebrew-installed LLVM is present #19413

Closed BenWibking closed 6 months ago

BenWibking commented 7 months ago

Describe the bug

Compiling VisIt 3.4.1RC on macOS 14 fails when building qt6.

This appears to be because if LLVM is installed via Homebrew, build_visit uses libtool from the Homebrew installation, rather than from XCode.

This may be the same issue: https://github.com/telegramdesktop/tdesktop/issues/25058

To Reproduce

Steps to reproduce the behavior. For example:

  1. Run ./build_visit --qt6
  2. See error: 
    $ ./build_visit --qt6
Processing bsd license.
bv_vtk_initialize
bv_python_info
bv_vtk_info
setting up vtk for version 9
[build_visit invocation arguments] --qt6
[build_visit build mode ] Release
apple clang version 17.0.6
Fortran support for thirdparty libraries disabled.
enabling any dependent libraries
The third party library location, "./third_party" does not exist. Create it? [yes/no]
yes
[...]
Building QT6 base (~10 minutes)
Unzipping/Untarring qtbase-everywhere-src-6.4.2.tar.xz . . .
Patching qt6 . . .
Patching qt 6 for macOS c++ stdlib issue
Configuring Qt6 base: . . .
Building Qt6 base . . .
Qt6 base build failed. Giving up
Unable to build or install QT6. Bailing out.
Error in build process. See build_visit_log for more information. If the error is unclear, please include build_visit_log and contact the VisIt project via https://visit-help.llnl.gov. You may need to compress the build_visit_log using a program like gzip so it will fit within the size limits for attachments.
Log file full path:  /Users/benwibking/visit/src/tools/dev/scripts/build_visit_log

The actual error from the logfile is:

/opt/homebrew/opt/llvm/bin/llvm-ranlib: error: Invalid option: '-no_warning_for_no_symbols'

Desktop

Additional context

Logfile: build_visit_log.zip

markcmiller86 commented 7 months ago

@BenWibking whenever I build on a mac with "special stuff" installed, I need to take explicit action to prevent build_visit from using any of that. To do that, I do something like...

env PATH=/usr/local/bin:/usr/bin:/bin:/sbin build_visit
cyrush commented 6 months ago

Not sure how we can defend against this - other than requesting that homebrew is not in PATH before using build_visit. Should we document this in some way?

BenWibking commented 6 months ago

Not sure how we can defend against this - other than requesting that homebrew is not in PATH before using build_visit. Should we document this in some way?

Maybe it could check whether the compiler identifies itself as "Apple clang" (and abort if not) when building on Mac?

$  ~ env PATH=/usr/local/bin:/usr/bin:/bin:/sbin clang -v
Apple clang version 15.0.0 (clang-1500.1.0.2.5)
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$  ~ clang -v
Homebrew clang version 17.0.6
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin
BenWibking commented 6 months ago

@BenWibking whenever I build on a mac with "special stuff" installed, I need to take explicit action to prevent build_visit from using any of that. To do that, I do something like...

env PATH=/usr/local/bin:/usr/bin:/bin:/sbin build_visit

Strangely, I can't get it to build qwt with this command. If I do brew uninstall llvm and then build, it works, but setting the above PATH leads to a strange compiler error when building qwt.

cyrush commented 6 months ago

Can you share the qwt build issue?

We are in a constant battle with packages that poke around the file system and find random installs. We try to be as explicit as we can about pointing our builds to our TPLs to avoid this, but is a challenge.

BenWibking commented 6 months ago

@cyrush This is the log for the qwt build error: build_visit_log.zip

I built with: env PATH=/usr/local/bin:/usr/bin:/bin:/sbin ./build_visit --qt6 --boxlib --adios2 --blosc2

I used the rel-mcm86-22mar24-more-341-darwin-fixes branch with the following diff (copied from the main branch):

diff --git a/src/tools/dev/scripts/bv_support/bv_main.sh b/src/tools/dev/scripts/bv_support/bv_main.sh
index c0f16d5988..cd5f70401a 100644
--- a/src/tools/dev/scripts/bv_support/bv_main.sh
+++ b/src/tools/dev/scripts/bv_support/bv_main.sh
@@ -365,6 +365,14 @@ function initialize_build_visit()
             export MACOSX_DEPLOYMENT_TARGET=11.0
         elif [[ ${VER_MAJOR} == 21 ]] ; then
             export MACOSX_DEPLOYMENT_TARGET=12.0
+        elif [[ ${VER_MAJOR} == 22 ]] ; then
+            export MACOSX_DEPLOYMENT_TARGET=13.0
+        elif [[ ${VER_MAJOR} == 23 ]] ; then
+            # keep 13 (ventura)
+            export MACOSX_DEPLOYMENT_TARGET=13.0
+        elif [[ ${VER_MAJOR} == 24 ]] ; then
+            # keep 13 (ventura)
+            export MACOSX_DEPLOYMENT_TARGET=13.0
         else
             echo "Unsupported Darwin major version, ${VER_MAJOR}."
             echo "Maybe add a new case for MACOSX_DEPLOYMENT_TARGET"
BenWibking commented 6 months ago

Just after I posted that, I did some more digging and I think it's a CMake environment variable issue. Testing a possible fix now.

BenWibking commented 6 months ago

Ok, that was caused by having CMAKE_GENERATOR set in my environment.

I fixed that, but now I'm getting a strange, apparently unrelated issue building Python:

checking if python is installed
python is not installed
Building Python (~3 minutes)
prepare_build_dir: Python-3.9.18 Python-3.9.18.tgz
Found Python-3.9.18 . . .
Configuring Python
+ ./configure 'OPT=-fno-common -fexceptions -O2' CXX=clang++ CC=clang 'LDFLAGS=-Wl,-rpath,/Users/benwibking/visIt/src/tools/dev/scripts/third_party/python/3.9.18/arm-apple-darwin23_clang/lib/ -pthread -L/Users/benwibking/visIt/src/tools/dev/scripts/third_party/zlib/1.2.13/arm-apple-darwin23_clang/lib' 'CPPFLAGS= -I/Users/benwibking/visIt/src/tools/dev/scripts/third_party/zlib/1.2.13/arm-apple-darwin23_clang/include' --enable-shared --prefix=/Users/benwibking/visIt/src/tools/dev/scripts/third_party/python/3.9.18/arm-apple-darwin23_clang --disable-ipv6
./bv_support//bv_python.sh: line 626: ./configure: No such file or directory
+ set +x
Building Python . . . (~3 minutes)
make: *** No targets specified and no makefile found.  Stop.
Python build failed. Giving up
Unable to build or install Python. Bailing out.
Error in build process. See /Users/benwibking/visIt/src/tools/dev/scripts/build_visit_log for more information. If the error is unclear, please include /Users/benwibking/visIt/src/tools/dev/scripts/build_visit_log and contact the VisIt project via https://visit-help.llnl.gov. You may need to compress the /Users/benwibking/visIt/src/tools/dev/scripts/build_visit_log using a program like gzip so it will fit within the size limits for attachments.
Log file full path:  /Users/benwibking/visIt/src/tools/dev/scripts//Users/benwibking/visIt/src/tools/dev/scripts/build_visit_log

Full log: build_visit_log.zip

JustinPrivitera commented 6 months ago

The log doesn't suggest to me what the issue might be. In absence of another VisIt team member providing a different path forward, I would suggest starting a clean build if you haven't already. Sometimes when libraries fail to build and you restart, VisIt thinks the ones that failed actually succeeded and keeps chugging along, leading to mysterious errors further in the pipeline.

BenWibking commented 6 months ago

The log doesn't suggest to me what the issue might be. In absence of another VisIt team member providing a different path forward, I would suggest starting a clean build if you haven't already. Sometimes when libraries fail to build and you restart, VisIt thinks the ones that failed actually succeeded and keeps chugging along, leading to mysterious errors further in the pipeline.

Ah... that fixed it. I had to re-clone the repo, as git clean was not sufficient. This might also have caused the qwt build issue.

JustinPrivitera commented 6 months ago

I'm glad. :) Dependencies can end up in our third party directory even if they fail to build, which confuses build_visit. Sorry about that and glad you are set. Can we close the ticket?

markcmiller86 commented 6 months ago

So, I have homebrew on my system and I wind up having to go to great lenghts to ensure build_visit doesn't find it. So, I do something like..

env PATH=/usr/local/bin:/usr/bin:/bin:/sbin build_visit

And, that only partially works because several of the packages build_visit builds go off looking for stuff in common places and then finding things I don't want. Then, I have to go into those specific packages and find the knobs to disable that search or to force it to find the stuff where I want it to find it.

All that said, I am working on an update to build_visit in #19412 that appears to work in the presence of homebrew on my Intel macOS 12.7.4 system.

BenWibking commented 6 months ago

I'm glad. :) Dependencies can end up in our third party directory even if they fail to build, which confuses build_visit. Sorry about that and glad you are set. Can we close the ticket?

Yes, the build finally finished successfully (using env PATH=/usr/local/bin:/usr/bin:/bin:/sbin ./build_visit --qt6 --boxlib --adios2 --blosc2 when starting from scratch).

BenWibking commented 6 months ago

Since things appear to compile on my system now, I'll close this ticket. However, I am still seeing the runtime issue in https://github.com/visit-dav/visit/issues/19415 that prevents the main viewer windows from opening.