ceheitsch
commented
3 years ago Open ceheitsch opened 3 years ago
ceheitsch
commented
3 years ago ceheitsch
commented
3 years ago
@(#)PROGRAM:ld PROJECT:ld64-450.3
BUILD 18:01:43 Mar 13 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 10.0.1, (clang-1001.0.46.3) (static support for 22, runtime is 22)
TAPI support using: Apple TAPI version 10.0.1 (tapi-1001.0.4.1)```
maxieds
commented
3 years ago @ceheitsch
I updated the C++ standard to V14. Please try git pull, then retry the cmake followed by make procedure.
ceheitsch
commented
3 years ago Alas; no luck.
[ 1%] Linking CXX executable bin/gtmfe
/usr/local/Cellar/cmake/3.19.4/bin/cmake -E cmake_link_script CMakeFiles/gtmfe.dir/link.txt --verbose=1
/usr/local/opt/llvm/bin/clang++ -Wall -mlinker-version=450 -std=c++14 -fPIC -fopenmp -g -O2 -fno-lto -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/ -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include -DGTFOLD_PACKAGE_VERSION="1.0.0" -Wall -mlinker-version=450 -std=c++14 -fPIC -fopenmp -g -O2 -fno-lto -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/ -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include -DGTFOLD_PACKAGE_VERSION="1.0.0" -L/usr/local/opt/llvm/lib -mlinker-version=450 -L/usr/local/lib -lgmp -lgmpxx -lm -lpthread -L/usr/local/Cellar/gmp/6.2.1/lib -lgmp -L/usr/local/Cellar/gmp/6.2.1/lib -lgmpxx -lgmp -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -mmacosx-version-min=10.14 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wall -mlinker-version=450 -std=c++14 -fPIC -fopenmp -g -O2 -fno-lto -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/ -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include -DGTFOLD_PACKAGE_VERSION="1.0.0" -L/usr/local/opt/llvm/lib -mlinker-version=450 -L/usr/local/lib -lgmp -lgmpxx -lm -lpthread -L/usr/local/Cellar/gmp/6.2.1/lib -lgmp -L/usr/local/Cellar/gmp/6.2.1/lib -lgmpxx -lgmp CMakeFiles/gtmfe.dir/gtfold-mfe/src/AdvancedDouble.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/algorithms-partition.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/algorithms.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/boltzmann_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/constraints.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/energy.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/global.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/key.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/loader.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/mfe_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/options.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-dangle.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-func-d2.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-func.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/pf-shel-check.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/random-sample.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/shapereader.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/stochastic-sampling-d2.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/stochastic-sampling.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/subopt_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/subopt_traceback.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/traceback.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/utils.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/binexe-utils/main.cc.o -o bin/gtmfe
undef: _myExp
Undefined symbols for architecture x86_64:
"_myExp", referenced from:
_f in lto.o
_calc_up_parallel in lto.o
_calc_s1 in lto.o
_calc_s2 in lto.o
_calc_upm in lto.o
_calc_up in lto.o
_calc_s3 in lto.o
...
ld: symbol(s) not found for architecture x86_64
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [bin/gtmfe] Error 1
make[1]: *** [CMakeFiles/gtmfe.dir/all] Error 2
make: *** [all] Error 2@ceheitsch
Ok. It looks like the templated code defined in a header needs to get moved to a .cc file for the symbols to get recognized. I will work on this for the week.
maxieds
commented
3 years ago @ceheitsch If you still get errors, try running the following command first (see reference):
$ export CPATH=/Library/Developer/CommandLineTools/usr/include/c++/v1Another option seems to be trying (we can walk through this on on Thursday if it comes down to it):
$ sudo xcode-select -s /Applications/Xcode.app/Contents/DeveloperPlease let me know what if any of these options works.
ceheitsch
commented
3 years ago The 'export' option did not seem to make a difference. FWIW:
xcode-select -print-path
/Library/Developer/CommandLineToolsThere is no Xcode.app directory under Applications:
% xcode-select -s /Applications/Xcode.app/Contents/Developer
xcode-select: error: invalid developer directory '/Applications/Xcode.app/Contents/Developer'FYI, the account where the code is being compiled doesn't have sudo privileges. The machine admin is handled by a different account.
maxieds
commented
3 years ago @ceheitsch
It's actually preferable IMHO not to deal with the full XCode install anyway. Apple has this lovely tendency of selectively supporting, and then breaking their developer tools. This issue should boil down to a picky templates issue with changes in the C++ standards. It should not be as burdensome for GTFold users to install this on a Mac with cmake. Let me take some time to pick apart the code on another Mojave system and see if I can fix things without needing a critical upgrade to the vendor sanctioned toolchain.
ceheitsch
commented
3 years ago @maxieds Your approach makes a lot of sense. Unfortunately, the fresh 'make' didn't work, but it failed in a. different. way (so I'm counting that as progress). I'm attaching the whole output so you have all that info. output_feb24.txt
maxieds
commented
3 years ago @ceheitsch
This reminds me of a longstanding fix for the non-standard cstring support on MacOS from RNAStructViz. Let me look at the errors more closely today. I will see if I can find a patch for the issue. This is actually encouraging in so much as it will probably just require adding in the include definitions for libc/libstdc++ from within the newly installed llvm sources.
This link on StackOverflow (for Catalina/10.15) should give you more perspective on how painful it is to maintain the compiler tools on Mac vendor systems without external software.
maxieds
commented
3 years ago @ceheitsch
Please try git pull && cmake [...] && make again where the arguments to cmake are documented as usual here.
I tried the following two fixes, both of which still generate a correctly compiling and linking build process on my Mac:
-isystem /usr/local/opt/llvm/include to the CMakeLists.txt configuration. The idea is to make sure that the compiler knows where to find its most up to date expected libc++ headers. This proposed fix should take into account some of my previous best guesses at the build problems related to problematic, or incorrectly symlinked, c++ headers on the system.clang++ compilers do not take well to the older style C header spec with #include <string.h>. I updated these references in the gtfold c++/*.cc/*.h sources to now point to #include <cstring>, a more modern standard way of getting to the same string functions. This seems to address the specific build errors you were having on Mojave (referenced in the link from my last reply).Please let me know if this works. Otherwise, we can iterate if new errors are invoked after fixing these ones.
ceheitsch
commented
3 years ago Attaching output.
output_feb24_2nd.txt
maxieds
commented
3 years ago @ceheitsch
Please rerun git pull followed by the build commands. The latest commit removes the libstdc++ includes from the default search path, and then carefully re-appends the corresponding header directories to include these files, e.g., #include <iostream>, back to the working search path. It's not pretty, but it continues to build on my Mojave box. I suspect you should minimally at least encounter new errors output by the build with this fix. Let me know if it works.
My (truncated after the gtmfe build and test cases) output from compiling is attached as a reference point:
issue2-working-output-2021-02-25.out.txt
ceheitsch
commented
3 years ago @maxieds Ran through the process. Output attached. output_feb26.txt
maxieds
commented
3 years ago @ceheitsch I have another round of proposed fixes for these weird build errors in the most recent commit.
ceheitsch
commented
3 years ago @maxieds I tried on a different mac that I have access to at home, and it too failed. Interestingly, it failed in the same way as the original problem reported at the beginning of this issue, i.e.
Undefined symbols for architecture x86_64:
"_myExp", referenced from:
_f in lto.o
_calc_up_parallel in lto.o
_calc_s1 in lto.o
_calc_s2 in lto.o
_calc_upm in lto.o
_calc_up in lto.o
_calc_s3 in lto.o
...
ld: symbol(s) not found for architecture x86_64FWIW, cmake was updated to 3.19.6, llvm 11.1.0 is now installed, and libomp was already updated at some point to 11.1.0.
I also note that it got about 20% through the make, which is considerably farther that we were getting on the other system.
ceheitsch
commented
3 years ago @maxieds On the original system (OSX 10.14.4), I tried a refresh install of llvm (11.1.0), cmake (3.19.6) and libomp (now 11.1.0). I also recloned the repo, just in case. Errors attached. output10.14.4_mar11.txt
ceheitsch
commented
3 years ago @maxieds Here is the output on the other system (OSX 10.14.6).
output10.14.6_mar10.txt
ceheitsch
commented
3 years ago SoM IT installed cmake on my office machine, and gtfold builds under my account there. FWIW, I noted that clang-9 is being used, and not clang-11 as it is on the home machine(s).
maxieds
commented
3 years ago @ceheitsch
Unless I actually introduced new errors when we were debugging yesterday online, this error should have been fixed long ago. Please make sure you have recently run git pull, followed by the cmake (...) command, and then finally followed by make.
I had to go in and change some of the includes in some of the C/C++ source files so that they are now compatible with a modern clang++ compiler. It has been several weeks since this showed up last. If that doesn't work, make sure that the paths to llvm are correct. I recall that if the stock MacOS system ones get loaded before the updated ones, there will be issues.
ceheitsch
commented
3 years ago @maxieds That is exactly the point. I get the same error on a different system so clearly whatever was "fixed" previously was too localized.
ceheitsch
commented
3 years ago Is this relevant? https://github.com/Homebrew/homebrew-core/issues/45061
ceheitsch
commented
3 years ago @maxieds I seem to recall asking about paths before. My memory is that you assured me that this wouldn't matter since they would be call be absolute paths.
maxieds
commented
3 years ago @ceheitsch
I looked at the CMakeLists.txt configuration after I noticed that the gtfold files were building with the custom Makefile I wrote last year for the Python bindings. I found something interesting:
$ python3-config --embed --cflags
-I/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9
-I/usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9
-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common
-dynamic -DNDEBUG -g -fwrapv -O3 -Wall
-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdkNote that if we do not call python3-config --libs --ldflags (as a convenience since it knows where to find things better on the system that I do, not because gtfold needs python), then this gets appended and everything goes to he-double-hockey-sticks:
-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk
-mmacosx-version-min=10.14I am led to conclude that some of the issues we have been having on the different Mac Mojave systems is related to which version of the platform SDK is installed, and where it is located. (Working, but good intuitive hypothesis)
maxieds
commented
3 years ago Note also, that 10.15 is notably broken. So if the problem is that some installs point to a bad SDK, that could be a way to pinpoint build problems.
maxieds
commented
3 years ago @ceheitsch It appears that the following procedure should fix things:
$ sudo xcode-select -s /Library/Developer/CommandLineTools
# Then add the following to the CMake command:
#-DMAC_SYSROOT_PATH="/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk"The updated docs are at the usual location.