search

StanfordLegion / legion

The Legion Parallel Programming System
https://legion.stanford.edu
Apache License 2.0
680 stars 145 forks source link

Missing Dependency in Quickstart for Ubuntu #618

Closed qyz96 closed 5 years ago

qyz96 commented 5 years ago

Hi,
I am having the same error as the one in https://github.com/StanfordLegion/legion/issues/361 The exact error is [adncat@sh-116-06 ~/legion/language]$ ./install.py --debug Already up-to-date. mkdir -p release/bin release/lib g++ -shared -fPIC -Wl,-export-dynamic -Wl,--whole-archive release/lib/libterra.a -Wl,--no-whole-archive -lcurses -lz -ldl -pthread -o release/lib/terra.so
g++ build/main.o build/linenoise.o -o release/bin/terra -g -Wl,-export-dynamic -Wl,--whole-archive release/lib/libterra.a -Wl,--no-whole-archive -lcurses -lz -ldl -pthread /usr/bin/ld: cannot find -lcurses collect2: error: ld returned 1 exit status make: [release/bin/terra] Error 1 make: Waiting for unfinished jobs.... /usr/bin/ld: cannot find -lcurses collect2: error: ld returned 1 exit status make: * [release/lib/terra.so] Error 1 Traceback (most recent call last): File "./install.py", line 437, in driver() File "./install.py", line 434, in driver install(vars(args)) File "./install.py", line 340, in install thread_count, llvm) File "./install.py", line 183, in install_terra build_terra(terra_dir, terra_branch, thread_count, llvm) File "./install.py", line 137, in build_terra cwd=terra_dir) File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['make', 'all', '-j', '24']' returned non-zero exit status 2

However, I am running the code on the school cluster so I cannot use the sudo apt-get command in the previous issue. Is there another way to fix this?

elliottslaughter commented 5 years ago

It might take a while to complete, but scripts/setup_env.py is the best way to get a self-consistent set of libraries (LLVM, etc) with minimal dependencies. There is the an issue with auto-detection in some cases, but unless you want to debug that yourself, it's probably easier to use setup_env.py.

streichler commented 5 years ago

@qyz96 were you able to get things to work with setup_env.py?

qyz96 commented 5 years ago

I am trying to figure out how to fix the following error: [adncat@sh-116-05 ~/legion/language]$ ./scripts/setup_env.py Traceback (most recent call last): File "./scripts/setup_env.py", line 526, in driver(**vars(args)) File "./scripts/setup_env.py", line 339, in driver raise Exception('Please set CC in your environment') Exception: Please set CC in your environment

aheirich commented 5 years ago

Try export CC=gcc, export CXX=g++

On Oct 8, 2019, at 9:40 AM, qyz96 notifications@github.com<mailto:notifications@github.com> wrote:

I am trying to figure out how to fix the following error: [adncat@sh-116-05 ~/legion/language]$ ./scripts/setup_env.py Traceback (most recent call last): File "./scripts/setup_env.py", line 526, in driver(**vars(args)) File "./scripts/setup_env.py", line 339, in driver raise Exception('Please set CC in your environment') Exception: Please set CC in your environment

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/StanfordLegion/legion/issues/618?email_source=notifications&email_token=AAORXXWF5CB47ABB7D5ZO3LQNSZXBA5CNFSM4I5VDCS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAU2F5Y#issuecomment-539599607, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAORXXUJ6M55A7GZOI4MN33QNSZXBANCNFSM4I5VDCSQ.

qyz96 commented 5 years ago

It seems that there are other variables not set in the environment: [adncat@sh-116-05 ~/legion/language]$ ./scripts/setup_env.py Cloning into '/home/users/adncat/legion/language/gasnet'... remote: Enumerating objects: 27, done. remote: Counting objects: 100% (27/27), done. remote: Compressing objects: 100% (20/20), done. remote: Total 137 (delta 12), reused 21 (delta 6), pack-reused 110 Receiving objects: 100% (137/137), 18.78 MiB | 0 bytes/s, done. Resolving deltas: 100% (73/73), done. Traceback (most recent call last): File "./scripts/setup_env.py", line 526, in driver(**vars(args)) File "./scripts/setup_env.py", line 391, in driver conduit = discover_conduit() File "./scripts/setup_env.py", line 51, in discover_conduit raise Exception('Please set CONDUIT in your environment') Exception: Please set CONDUIT in your environment

elliottslaughter commented 5 years ago

Yes, if you're on a machine that we haven't specifically tested for in the past, you will need to configure some options for setup_env.py. These will include CC and CXX as Alan noted as well as CONDUIT, which is the name of the network to use (e.g. ibv for infiniband). Based on your login prompt it sounds like you may be on Sherlock, in which case ibv is the correct choice.

qyz96 commented 5 years ago

Yes, after setting the CONDUIT variable I can run the script now, but there seems to be some problems when building llvm:

[ 19%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseAST.cpp.o [ 19%] Building CXX object tools/clang/lib/ASTMatchers/CMakeFiles/clangASTMatchers.dir/ASTMatchFinder.cpp.o [ 19%] Building CXX object tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/Diagnostics.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/APValue.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTConsumer.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTContext.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTDiagnostic.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTDumper.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CodeGenRegisters.cpp.o] Error 4 make[2]: Waiting for unfinished jobs.... [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTImporter.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTStructuralEquivalence.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTTypeTraits.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/AttrImpl.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/CXXInheritance.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/Comment.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/GlobalISelEmitter.cpp.o] Error 4 [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/CommentBriefParser.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/CommentCommandTraits.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/CommentLexer.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Builtins.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/RegisterInfoEmitter.cpp.o] Error 4 [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/CharInfo.cpp.o make[1]: [utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/all] Error 2 make[1]: Waiting for unfinished jobs.... [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Cuda.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/CommentParser.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseAST.cpp.o] Error 4 make[1]: [tools/clang/lib/Parse/CMakeFiles/clangParse.dir/all] Error 2 [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/CommentSema.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Diagnostic.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [tools/clang/lib/ASTMatchers/CMakeFiles/clangASTMatchers.dir/ASTMatchFinder.cpp.o] Error 4 make[1]: [tools/clang/lib/ASTMatchers/CMakeFiles/clangASTMatchers.dir/all] Error 2 [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/DiagnosticIDs.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/DiagnosticOptions.cpp.o [ 19%] Building CXX object tools/clang/lib/AST/CMakeFiles/clangAST.dir/DataCollection.cpp.o [ 19%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/Lexer.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/FileManager.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/Diagnostics.cpp.o] Error 4 make[1]: [tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/all] Error 2 [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/FileSystemStatCache.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/IdentifierTable.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [tools/clang/lib/AST/CMakeFiles/clangAST.dir/APValue.cpp.o] Error 4 make[2]: Waiting for unfinished jobs.... [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/LangOptions.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/MemoryBufferCache.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTDiagnostic.cpp.o] Error 4 [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Module.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTContext.cpp.o] Error 4 [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/ObjCRuntime.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/OpenMPKinds.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/OperatorPrecedence.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/SanitizerBlacklist.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTDumper.cpp.o] Error 4 [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/SanitizerSpecialCaseList.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Sanitizers.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://gcc.gnu.org/bugs.html for instructions. make[2]: [tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTImporter.cpp.o] Error 4 [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/SourceLocation.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/SourceManager.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/TargetInfo.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/AArch64.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/AMDGPU.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/ARM.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/AVR.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/BPF.cpp.o [ 19%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/Hexagon.cpp.o [ 19%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/LiteralSupport.cpp.o [ 19%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/MacroArgs.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/Lanai.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/Le64.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/MSP430.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/Mips.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/NVPTX.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/Nios2.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/OSTargets.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/PNaCl.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/PPC.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/SPIR.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/Sparc.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/SystemZ.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/TCE.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/MacroInfo.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/ModuleMap.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PPCaching.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/WebAssembly.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/X86.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Targets/XCore.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/TokenKinds.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Version.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/VersionTuple.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/VirtualFileSystem.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/Warnings.cpp.o [ 21%] Building CXX object tools/clang/lib/Basic/CMakeFiles/clangBasic.dir/XRayLists.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PPCallbacks.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PPConditionalDirectiveRecord.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PPDirectives.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PPExpressions.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PPLexerChange.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PPMacroExpansion.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PTHLexer.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/Pragma.cpp.o make[1]: [tools/clang/lib/AST/CMakeFiles/clangAST.dir/all] Error 2 [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PreprocessingRecord.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/Preprocessor.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/PreprocessorLexer.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/ScratchBuffer.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/TokenConcatenation.cpp.o [ 21%] Building CXX object tools/clang/lib/Lex/CMakeFiles/clangLex.dir/TokenLexer.cpp.o [ 21%] Linking CXX static library ../../../../lib/libclangBasic.a [ 21%] Built target clangBasic [ 21%] Linking CXX static library ../../../../lib/libclangLex.a [ 21%] Built target clangLex make: [all] Error 2

####################################################################

Build Failed

####################################################################

It appears that llvm has failed to build. The failure was:

Traceback (most recent call last): File "./scripts/setup_env.py", line 447, in driver install_llvm(llvm_dir, llvm_install_dir, scratch_dir, llvm_version, llvm_use_cmake, cmake_exe, thread_count, cache, is_cray, insecure) File "./scripts/setup_env.py", line 273, in install_llvm build_llvm(llvm_source_dir, llvm_build_dir, llvm_install_dir, llvm_use_cmake, cmake_exe, thread_count, is_cray) File "./scripts/setup_env.py", line 133, in build_llvm subprocess.check_call(['make', '-j', str(thread_count)], cwd=build_dir) File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['make', '-j', '20']' returned non-zero exit status 2

elliottslaughter commented 5 years ago

If this is Sherlock, you should know that the administrators place a memory limit on the head node, so your build will get killed like this if you try to run directly on the head node. You can build on a compute node with the appropriate salloc line and it should have enough memory to go through.

qyz96 commented 5 years ago

Yes this is Sherlock. I used the salloc command and ran setup_env.py on a compute node. I still encountered similar errors: [ 22%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseObjc.cpp.o [ 22%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseOpenMP.cpp.o [ 22%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParsePragma.cpp.o [ 22%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseStmt.cpp.o [ 22%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseStmtAsm.cpp.o [ 22%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseTemplate.cpp.o [ 22%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseTentative.cpp.o [ 24%] Building CXX object tools/clang/lib/Parse/CMakeFiles/clangParse.dir/Parser.cpp.o [ 24%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/BodyFarm.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://bugzilla.redhat.com/bugzilla for instructions. make[2]: [tools/clang/lib/ASTMatchers/CMakeFiles/clangASTMatchers.dir/ASTMatchFinder.cpp.o] Error 4 make[1]: [tools/clang/lib/ASTMatchers/CMakeFiles/clangASTMatchers.dir/all] Error 2 make[1]: Waiting for unfinished jobs.... [ 24%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/CFG.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://bugzilla.redhat.com/bugzilla for instructions. make[2]: [tools/clang/lib/AST/CMakeFiles/clangAST.dir/ASTContext.cpp.o] Error 4 make[1]: [tools/clang/lib/AST/CMakeFiles/clangAST.dir/all] Error 2 [ 24%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/CFGReachabilityAnalysis.cpp.o [ 24%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/CFGStmtMap.cpp.o [ 24%] Linking CXX static library ../../../../lib/libclangBasic.a [ 24%] Built target clangBasic [ 24%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/CallGraph.cpp.o [ 24%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/CloneDetection.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://bugzilla.redhat.com/bugzilla for instructions. make[2]: [tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/Registry.cpp.o] Error 4 make[2]: Waiting for unfinished jobs.... [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/CocoaConventions.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/Consumed.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://bugzilla.redhat.com/bugzilla for instructions. make[2]: [tools/clang/lib/Sema/CMakeFiles/clangSema.dir/AnalysisBasedWarnings.cpp.o] Error 4 make[1]: [tools/clang/lib/Sema/CMakeFiles/clangSema.dir/all] Error 2 [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/CodeInjector.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/Dominators.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://bugzilla.redhat.com/bugzilla for instructions. make[2]: [tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/Parser.cpp.o] Error 4 make[1]: [tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/all] Error 2 [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/FormatString.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://bugzilla.redhat.com/bugzilla for instructions. make[2]: [tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseExpr.cpp.o] Error 4 make[2]: Waiting for unfinished jobs.... [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/LiveVariables.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://bugzilla.redhat.com/bugzilla for instructions. make[2]: [tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseExprCXX.cpp.o] Error 4 [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/OSLog.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/ObjCNoReturn.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/PostOrderCFGView.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/PrintfFormatString.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/ProgramPoint.cpp.o g++: internal compiler error: Killed (program cc1plus) Please submit a full bug report, with preprocessed source if appropriate. See http://bugzilla.redhat.com/bugzilla for instructions. make[2]: [tools/clang/lib/Parse/CMakeFiles/clangParse.dir/ParseStmtAsm.cpp.o] Error 4 [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/PseudoConstantAnalysis.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/ReachableCode.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/ScanfFormatString.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/ThreadSafety.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/ThreadSafetyCommon.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/ThreadSafetyLogical.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/ThreadSafetyTIL.cpp.o [ 25%] Building CXX object tools/clang/lib/Analysis/CMakeFiles/clangAnalysis.dir/UninitializedValues.cpp.o make[1]: [tools/clang/lib/Parse/CMakeFiles/clangParse.dir/all] Error 2 [ 25%] Linking CXX static library ../../../../lib/libclangAnalysis.a [ 25%] Built target clangAnalysis make: *** [all] Error 2

####################################################################

Build Failed

####################################################################

It appears that llvm has failed to build. The failure was:

Traceback (most recent call last): File "./scripts/setup_env.py", line 447, in driver install_llvm(llvm_dir, llvm_install_dir, scratch_dir, llvm_version, llvm_use_cmake, cmake_exe, thread_count, cache, is_cray, insecure) File "./scripts/setup_env.py", line 273, in install_llvm build_llvm(llvm_source_dir, llvm_build_dir, llvm_install_dir, llvm_use_cmake, cmake_exe, thread_count, is_cray) File "./scripts/setup_env.py", line 133, in build_llvm subprocess.check_call(['make', '-j', str(thread_count)], cwd=build_dir) File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['make', '-j', '20']' returned non-zero exit status 2

Given the number of things that could potentially have gone wrong, this script is not designed to handle this situation. You will need to fix the problem on your own (or ask for help fixing it).

The files are located here:

/home/users/adncat/legion/language/llvm

Once you have fixed the problem, you have two options:

  1. Go to the directory and rebuild it yourself. This script will not touch the directory again.

  2. Remove the directory. Then rerun this script.

Good luck and please ask for help if you get stuck!

elliottslaughter commented 5 years ago

If you're on Sherlock, salloc isn't actually enough to put you on a compute node. You will need to do something like:

salloc --partition=aaiken --tasks 1 --nodes=1 --cpus-per-task=20 --time=02:00:00
srun --pty bash

And you'll know you're on a compute node because the prompt will change to something like sh-114-01 instead of sh-ln04.

(Unfortunately this is machine specific so it's hard to give this advice in general; I only know this because I've used Sherlock myself.)

qyz96 commented 5 years ago

I am running the script on a gpu node: [adncat@sh-116-06 ~/legion/language]$ nvidia-smi Wed Oct 9 16:00:24 2019 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 418.87.01 Driver Version: 418.87.01 CUDA Version: 10.1 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Tesla V100-PCIE... On | 00000000:3F:00.0 Off | 0 | | N/A 30C P0 25W / 250W | 0MiB / 32480MiB | 0% E. Process | +-------------------------------+----------------------+----------------------+ | 1 Tesla V100-PCIE... On | 00000000:40:00.0 Off | 0 | | N/A 31C P0 24W / 250W | 0MiB / 32480MiB | 0% E. Process | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+

I still encountered an out of memory issue. By using the command dmesg, I got the following trace back:

[11548.111703] Task in /slurm/uid_332807/job_52376688/step_0/task_0 killed as a result of limit of /slurm/uid_332807/job_52376688/step_0 [11548.123690] memory: usage 8148992kB, limit 8148992kB, failcnt 7527152 [11548.130123] memory+swap: usage 8148992kB, limit 8148992kB, failcnt 0 [11548.136461] kmem: usage 0kB, limit 9007199254740988kB, failcnt 0 [11548.142453] Memory cgroup stats for /slurm/uid_332807/job_52376688/step_0: cache:0KB rss:0KB rss_huge:0KB mapped_file:0KB swap:0KB inactive_anon:0KB active_anon:0KB inactive_file:0KB active_file:0KB unevictable:0KB [11548.165627] Memory cgroup stats for /slurm/uid_332807/job_52376688/step_0/task_0: cache:1932KB rss:8147060KB rss_huge:0KB mapped_file:316KB swap:0KB inactive_anon:967304KB active_anon:7179756KB inactive_file:240KB active_file:1444KB unevictable:0KB [11548.191358] [ pid ] uid tgid total_vm rss nr_ptes swapents oom_score_adj name [11548.199302] [51238] 332807 51238 28802 962 13 0 0 bash [11548.207207] [51454] 332807 51454 55086 8581 65 0 0 python [11548.215305] [62525] 332807 62525 27560 739 12 0 0 make [11548.223208] [62530] 332807 62530 29383 2602 15 0 0 make [11548.231111] [66783] 332807 66783 27603 806 12 0 0 make [11548.239013] [66812] 332807 66812 27483 655 12 0 0 make [11548.246913] [67113] 332807 67113 27419 633 13 0 0 make [11548.254808] [67170] 332807 67170 27328 561 12 0 0 make [11548.262712] [68222] 332807 68222 28334 371 14 0 0 sh [11548.270439] [68223] 332807 68223 27967 160 11 0 0 g++ [11548.278260] [68224] 332807 68224 181297 143804 306 0 0 cc1plus [11548.286420] [68244] 332807 68244 28334 373 13 0 0 sh [11548.294144] [68245] 332807 68245 27967 160 12 0 0 g++ [11548.301955] [68246] 332807 68246 177633 140678 299 0 0 cc1plus [11548.310115] [68270] 332807 68270 28334 371 14 0 0 sh [11548.317838] [68271] 332807 68271 27967 160 13 0 0 g++ [11548.325649] [68273] 332807 68273 154241 116812 254 0 0 cc1plus [11548.333809] [68274] 332807 68274 28334 371 13 0 0 sh [11548.341532] [68275] 332807 68275 27967 160 11 0 0 g++ [11548.349344] [68276] 332807 68276 160407 122080 264 0 0 cc1plus [11548.357503] [68293] 332807 68293 28334 372 13 0 0 sh [11548.365229] [68294] 332807 68294 27967 160 12 0 0 g++ [11548.373040] [68295] 332807 68295 158196 119908 257 0 0 cc1plus [11548.381200] [68319] 332807 68319 28334 370 13 0 0 sh [11548.388934] [68320] 332807 68320 27967 159 12 0 0 g++ [11548.396745] [68321] 332807 68321 155014 116773 253 0 0 cc1plus [11548.404912] [68325] 332807 68325 28334 372 14 0 0 sh [11548.412635] [68326] 332807 68326 27967 159 12 0 0 g++ [11548.420453] [68327] 332807 68327 150105 112240 244 0 0 cc1plus [11548.428615] [68335] 332807 68335 28334 372 13 0 0 sh [11548.436353] [68336] 332807 68336 27967 161 13 0 0 g++ [11548.444169] [68337] 332807 68337 149194 110984 242 0 0 cc1plus [11548.452337] [68354] 332807 68354 28334 371 13 0 0 sh [11548.460060] [68356] 332807 68356 27967 159 13 0 0 g++ [11548.467883] [68357] 332807 68357 28334 371 12 0 0 sh [11548.475606] [68358] 332807 68358 143661 105898 232 0 0 cc1plus [11548.483774] [68359] 332807 68359 27967 159 11 0 0 g++ [11548.491582] [68360] 332807 68360 143916 105852 232 0 0 cc1plus [11548.499738] [68369] 332807 68369 28334 371 13 0 0 sh [11548.507466] [68371] 332807 68371 27967 160 13 0 0 g++ [11548.515276] [68372] 332807 68372 140818 104291 230 0 0 cc1plus [11548.523435] [68442] 332807 68442 28334 371 13 0 0 sh [11548.531159] [68443] 332807 68443 27967 159 12 0 0 g++ [11548.538973] [68444] 332807 68444 129209 91385 201 0 0 cc1plus [11548.547137] [68448] 332807 68448 28334 371 13 0 0 sh [11548.554862] [68449] 332807 68449 27967 160 12 0 0 g++ [11548.562675] [68450] 332807 68450 126416 87717 194 0 0 cc1plus [11548.570834] [68457] 332807 68457 28334 371 14 0 0 sh [11548.578569] [68458] 332807 68458 27967 160 11 0 0 g++ [11548.586379] [68459] 332807 68459 112975 78133 175 0 0 cc1plus [11548.594542] [68493] 332807 68493 28334 371 14 0 0 sh [11548.602270] [68494] 332807 68494 27967 160 12 0 0 g++ [11548.610084] [68495] 332807 68495 105786 71052 161 0 0 cc1plus [11548.618242] [68535] 332807 68535 28334 372 13 0 0 sh [11548.625976] [68537] 332807 68537 27967 159 11 0 0 g++ [11548.633786] [68538] 332807 68538 105944 70338 160 0 0 cc1plus [11548.641955] [68563] 332807 68563 28334 371 13 0 0 sh [11548.649677] [68564] 332807 68564 27967 160 12 0 0 g++ [11548.657489] [68565] 332807 68565 104354 66304 152 0 0 cc1plus [11548.665646] [68573] 332807 68573 28334 371 14 0 0 sh [11548.673375] [68574] 332807 68574 27967 160 12 0 0 g++ [11548.681185] [68575] 332807 68575 92383 57515 137 0 0 cc1plus [11548.689352] [68577] 332807 68577 28334 371 12 0 0 sh [11548.697074] [68578] 332807 68578 27967 160 13 0 0 g++ [11548.704887] [68579] 332807 68579 98589 60832 140 0 0 cc1plus [11548.713043] [68585] 332807 68585 28334 371 13 0 0 sh [11548.720770] [68586] 332807 68586 27967 160 12 0 0 g++ [11548.728581] [68587] 332807 68587 92823 54869 131 0 0 cc1plus [11548.736739] [68600] 332807 68600 28334 371 13 0 0 sh [11548.744465] [68601] 332807 68601 27967 160 12 0 0 g++ [11548.752275] [68602] 332807 68602 74817 36521 95 0 0 cc1plus [11548.760442] [68637] 332807 68637 28334 372 14 0 0 sh [11548.768176] [68638] 332807 68638 27967 159 12 0 0 g++ [11548.775988] [68639] 332807 68639 63180 25616 73 0 0 cc1plus [11548.784144] [68645] 332807 68645 28334 371 13 0 0 sh [11548.791869] [68646] 332807 68646 27967 160 12 0 0 g++ [11548.799685] [68647] 332807 68647 58860 20481 64 0 0 cc1plus [11548.807841] [68655] 332807 68655 28334 371 14 0 0 sh [11548.815566] [68656] 332807 68656 27967 160 12 0 0 g++ [11548.823378] [68657] 332807 68657 40217 5842 34 0 0 cc1plus [11548.831539] Memory cgroup out of memory: Kill process 68224 (cc1plus) score 70 or sacrifice child [11548.840384] Killed process 68224 (cc1plus) total-vm:725188kB, anon-rss:574188kB, file-rss:1028kB, shmem-rss:0kB

elliottslaughter commented 5 years ago

Sherlock still sets a memory limit if you don't ask for all the cores on a node. Without seeing your salloc command I cannot comment on whether you may be hitting this or not.

But at any rate, since you're on Sherlock, it may be easier just to load the existing modules on Sherlock for LLVM and GASNet (llvm/3.8.1 and gasnet/1.30.0), and set export GASNET=/share/software/user/open/gasnet/1.30.0, and then build using LD_FLAGS=-lpmi2 CONDUIT=ibv srun -c 20 ./install.py --gasnet or similar.

qyz96 commented 5 years ago

Yes I did not have any memory issue after using salloc --partition=aaiken --tasks 1 --nodes=1 --cpus-per-task=20 --time=02:00:00 But there still seems to be some compatibility errors when running setup_env.py rm -f liblegion.a ar rc liblegion.a /home/users/adncat/legion/runtime/legion/legion.cc.o /home/users/adncat/legion/runtime/legion/legion_c.cc.o /home/users/adncat/legion/runtime/legion/legion_ops.cc.o /home/users/adncat/legion/runtime/legion/legion_tasks.cc.o /home/users/adncat/legion/runtime/legion/legion_context.cc.o /home/users/adncat/legion/runtime/legion/legion_trace.cc.o /home/users/adncat/legion/runtime/legion/legion_spy.cc.o /home/users/adncat/legion/runtime/legion/legion_profiling.cc.o /home/users/adncat/legion/runtime/legion/legion_profiling_serializer.cc.o /home/users/adncat/legion/runtime/legion/legion_instances.cc.o /home/users/adncat/legion/runtime/legion/legion_views.cc.o /home/users/adncat/legion/runtime/legion/legion_analysis.cc.o /home/users/adncat/legion/runtime/legion/legion_constraint.cc.o /home/users/adncat/legion/runtime/legion/legion_mapping.cc.o /home/users/adncat/legion/runtime/legion/legion_redop.cc.o /home/users/adncat/legion/runtime/legion/region_tree.cc.o /home/users/adncat/legion/runtime/legion/runtime.cc.o /home/users/adncat/legion/runtime/legion/garbage_collection.cc.o /home/users/adncat/legion/runtime/legion/mapper_manager.cc.o /home/users/adncat/legion/runtime/mappers/default_mapper.cc.o /home/users/adncat/legion/runtime/mappers/mapping_utilities.cc.o /home/users/adncat/legion/runtime/mappers/shim_mapper.cc.o /home/users/adncat/legion/runtime/mappers/test_mapper.cc.o /home/users/adncat/legion/runtime/mappers/null_mapper.cc.o /home/users/adncat/legion/runtime/mappers/replay_mapper.cc.o /home/users/adncat/legion/runtime/mappers/debug_mapper.cc.o /home/users/adncat/legion/runtime/mappers/wrapper_mapper.cc.o ---> Linking objects into one binary: libregent.so mpicxx -o libregent.so regent.cc.o regent_partitions.cc.o bishop_mapper.cc.o bishop_c.cc.o murmur_hash3.cc.o -shared -Wl,--whole-archive -llegion -lrealm -Wl,--no-whole-archive -L. -llegion -lrealm -lrt -lpthread -ldl -rdynamic -L/home/users/adncat/legion/language/gasnet/release/lib -lrt -lm -lgasnet-ibv-par -libverbs -L/lib -lmpi -lz /usr/bin/ld: skipping incompatible /lib/librt.so when searching for -lrt /usr/bin/ld: skipping incompatible /lib/libpthread.so when searching for -lpthread /usr/bin/ld: skipping incompatible /lib/libdl.so when searching for -ldl /usr/bin/ld: skipping incompatible /lib/librt.so when searching for -lrt /usr/bin/ld: skipping incompatible /lib/libm.so when searching for -lm /usr/bin/ld: skipping incompatible /lib/libpthread.so when searching for -lpthread /usr/bin/ld: skipping incompatible /lib/libc.so when searching for -lc

elliottslaughter commented 5 years ago

Is the link step actually failling? It's a bit hard to tell from the output here.

qyz96 commented 5 years ago

My bad, the building is complete and I can run install.py successfully now.