search

ahmedbougacha / dagger

Binary Translator to LLVM IR
Other
216 stars 51 forks source link

out of memory when linking #21

Open huangshiyou opened 6 years ago

huangshiyou commented 6 years ago

My machine has 8GB of RAM and more than 100 GB of storage. But an error happens when linking llvm-lto,

collect2: fatal error: ld terminated with signal 9 [Killed] compilation terminated. tools/llvm-lto/CMakeFiles/llvm-lto.dir/build.make:264: recipe for target 'bin/llvm-lto' failed make[2]: [bin/llvm-lto] Error 1 make[2]: Deleting file 'bin/llvm-lto' CMakeFiles/Makefile2:15267: recipe for target 'tools/llvm-lto/CMakeFiles/llvm-lto.dir/all' failed make[1]: [tools/llvm-lto/CMakeFiles/llvm-lto.dir/all] Error 2 Makefile:149: recipe for target 'all' failed make: [all] Error 2

According to other people's experience online, this error was caused due to out of memory. But isn't 8GB sufficient for most software building?

tofes commented 6 years ago

I'm on master branch commit 75005d8b578c3c8aff75f87301895a2eb2ad04e5 Date: Sun Dec 3 06:50:35 2017 +0900 fix building (#20)

I increased my linux virtual machine memory to 55GB, and still got the error below:

[100%] Built target llvm-xray
/usr/bin/ld: final link failed: No space left on device
clang: error: linker command failed with exit code 1 (use -v to see invocation)
tools/llvm-dwp/CMakeFiles/llvm-dwp.dir/build.make:284: recipe for target 'bin/llvm-dwp' failed
make[2]: *** [bin/llvm-dwp] Error 1
CMakeFiles/Makefile2:18125: recipe for target 'tools/llvm-dwp/CMakeFiles/llvm-dwp.dir/all' failed
make[1]: *** [tools/llvm-dwp/CMakeFiles/llvm-dwp.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Built target llvm-lto2
/usr/bin/ld: final link failed: No space left on device
clang: error: linker command failed with exit code 1 (use -v to see invocation)
tools/opt/CMakeFiles/opt.dir/build.make:426: recipe for target 'bin/opt' failed
make[2]: *** [bin/opt] Error 1
CMakeFiles/Makefile2:20951: recipe for target 'tools/opt/CMakeFiles/opt.dir/all' failed
make[1]: *** [tools/opt/CMakeFiles/opt.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2

I don't think it's direct memory limit caused the error "final link failed: No space left on device" as a project this size should not take over 50GB of memory to build...I hope. I then increased my disk space (I had 11GB of free disk space), which finally build! So make sure you have enough disk space as well.

dagger-master/build$ make -j8
[  0%] Built target LLVMHello_exports
[  1%] Built target LLVMDemangle
[  1%] Built target llvm_vcsrevision_h
[  1%] Built target LLVMBinaryFormat
[  1%] Built target LLVMMCDisassembler
[  1%] Built target LLVMTableGen
[  1%] Built target LLVMMCParser
[  4%] Built target obj.llvm-tblgen
[  5%] Built target LLVMOption
[  5%] Built target LLVMMCAnalysis
[  5%] Built target LLVMDebugInfoMSF
[  5%] Built target LLVMObjectYAML
[  6%] Built target LLVMSymbolize
[  6%] Built target LLVMLineEditor
[  6%] Built target llvm-PerfectShuffle
[  6%] Built target gtest
[ 12%] Built target LLVMDebugInfoDWARF
[ 13%] Built target LLVMDebugInfoCodeView
[ 12%] Built target LLVMSupport
[ 13%] Built target count
[ 13%] Built target LTO_exports
[ 13%] Built target llvm-go
[ 16%] Built target LLVMMC
[ 16%] Built target BugpointPasses_exports
[ 17%] Built target LLVMFuzzerNoMainObjects
[ 17%] Built target llvm-mcmarkup
[ 17%] Built target SecondLib
[ 17%] Built target llvm-tblgen
[ 17%] Built target PipSqueak
[ 17%] Built target gtest_main
[ 17%] Built target LLVMFuzzer
[ 17%] Built target DynamicLibraryLib
[ 17%] Built target LLVMTestingSupport
[ 17%] Built target LLVMFuzzerNoMain
[ 17%] Built target yaml-bench
[ 17%] Built target FileCheck
[ 18%] Built target CvtResTableGen
[ 18%] Built target not
[ 18%] Built target llvm-config
[ 18%] Built target AttributeCompatFuncTableGen
[ 18%] Built target llvm-cxxfilt
[ 21%] Built target LLVMDebugInfoPDB
[ 21%] Built target intrinsics_gen
[ 21%] Built target LibOptionsTableGen
[ 21%] Built target LLVMIRReader
[ 21%] Built target LLVMLibDriver
[ 22%] Built target LLVMMIRParser
[ 22%] Built target LLVMBitReader
[ 22%] Built target LLVMBitWriter
[ 22%] Built target LLVMGlobalISel
[ 22%] Built target LLVMInstrumentation
[ 24%] Built target LLVMAsmPrinter
[ 25%] Built target LLVMSelectionDAG
[ 28%] Built target LLVMCore
[ 29%] Built target LLVMInstCombine
[ 29%] Built target LLVMHello
[ 29%] Built target LLVMVectorize
[ 29%] Built target LLVMLinker
[ 29%] Built target LLVMCoroutines
[ 29%] Built target LLVMObjCARCOpts
[ 32%] Built target LLVMTransformUtils
[ 32%] Built target LLVMExecutionEngine
[ 33%] Built target LLVMLTO
[ 33%] Built target LLVMInterpreter
[ 35%] Built target LLVMipo
[ 35%] Built target LLVMMCJIT
[ 36%] Built target LLVMObject
[ 36%] Built target LLVMOrcJIT
[ 36%] Built target LLVMRuntimeDyld
[ 36%] Built target LLVMTarget
[ 37%] Built target AArch64CommonTableGen
[ 37%] Built target AMDGPUCommonTableGen
[ 39%] Built target BPFCommonTableGen
[ 40%] Built target ARMCommonTableGen
[ 41%] Built target LanaiCommonTableGen
[ 43%] Built target HexagonCommonTableGen
[ 44%] Built target MipsCommonTableGen
[ 44%] Built target NVPTXCommonTableGen
[ 44%] Built target RISCVCommonTableGen
[ 45%] Built target PowerPCCommonTableGen
[ 45%] Built target MSP430CommonTableGen
[ 47%] Built target SparcCommonTableGen
[ 48%] Built target SystemZCommonTableGen
[ 48%] Built target XCoreCommonTableGen
[ 50%] Built target X86CommonTableGen
[ 50%] Built target LLVMAsmParser
[ 52%] Built target LLVMScalarOpts
[ 52%] Built target LLVMXRay
[ 54%] Built target LLVMProfileData
[ 54%] Built target LLVMCoverage
[ 54%] Built target LLVMPasses
[ 55%] Built target llvm-bcanalyzer
[ 55%] Built target llvm-cvtres
[ 55%] Built target llvm-dis
[ 55%] Built target llvm-diff
[ 55%] Built target llvm-opt-report
[ 55%] Built target llvm-dwarfdump
[ 55%] Built target LLVMDC
[ 55%] Built target llvm-size
[ 55%] Built target llvm-symbolizer
[ 55%] Built target llvm-strings
[ 55%] Built target llvm-readobj
[ 55%] Built target LLVMAArch64Info
[ 55%] Built target obj2yaml
[ 55%] Built target sanstats
[ 56%] Built target yaml2obj
[ 60%] Built target LLVMAnalysis
[ 62%] Built target llvm-pdbutil
[ 62%] Built target LLVMAArch64AsmParser
[ 63%] Built target LLVMAArch64AsmPrinter
[ 63%] Built target LLVMAArch64Disassembler
[ 63%] Built target LLVMAArch64Utils
[ 63%] Built target LLVMAMDGPUUtils
[ 63%] Built target LLVMAMDGPUInfo
[ 63%] Built target LLVMAArch64Desc
[ 63%] Built target LLVMAArch64DC
[ 63%] Built target LLVMARMInfo
[ 63%] Built target LLVMARMAsmParser
[ 63%] Built target LLVMAMDGPUDesc
[ 63%] Built target LLVMARMDisassembler
[ 64%] Built target LLVMARMAsmPrinter
[ 64%] Built target LLVMBPFDisassembler
[ 64%] Built target LLVMBPFAsmPrinter
[ 66%] Built target LLVMBPFInfo
[ 66%] Built target LLVMARMDesc
[ 66%] Built target LLVMBPFCodeGen
[ 66%] Built target LLVMBPFDesc
[ 66%] Built target LLVMHexagonAsmParser
[ 66%] Built target LLVMHexagonInfo
[ 66%] Built target LLVMHexagonDisassembler
[ 67%] Built target LLVMAArch64CodeGen
[ 67%] Built target LLVMLanaiInfo
[ 67%] Built target LLVMLanaiAsmParser
[ 67%] Built target LLVMHexagonDesc
[ 67%] Built target LLVMLanaiAsmPrinter
[ 74%] Built target LLVMCodeGen
[ 75%] Built target LLVMLanaiDesc
[ 75%] Built target LLVMLanaiDisassembler
[ 75%] Built target LLVMLanaiCodeGen
[ 75%] Built target LLVMMipsDisassembler
[ 75%] Built target LLVMMipsAsmPrinter
[ 77%] Built target LLVMARMCodeGen
[ 77%] Built target LLVMMipsInfo
[ 77%] Built target LLVMMSP430AsmPrinter
[ 77%] Built target LLVMMSP430Info
[ 77%] Built target LLVMMipsAsmParser
[ 77%] Built target LLVMMSP430Desc
[ 77%] Built target LLVMNVPTXInfo
[ 77%] Built target LLVMNVPTXAsmPrinter
[ 77%] Built target LLVMNVPTXDesc
[ 78%] Built target LLVMMipsDesc
[ 78%] Built target LLVMMSP430CodeGen
[ 78%] Built target LLVMPowerPCAsmParser
[ 78%] Built target LLVMPowerPCDisassembler
[ 78%] Built target LLVMPowerPCAsmPrinter
[ 78%] Built target LLVMPowerPCInfo
[ 78%] Built target LLVMRISCVInfo
[ 78%] Built target LLVMRISCVCodeGen
[ 78%] Built target LLVMPowerPCDesc
[ 81%] Built target LLVMHexagonCodeGen
[ 82%] Built target LLVMNVPTXCodeGen
[ 85%] Built target LLVMAMDGPUCodeGen
[ 85%] Built target LLVMRISCVDesc
[ 85%] Built target LLVMSparcInfo
[ 85%] Built target LLVMSparcAsmPrinter
[ 85%] Built target LLVMSparcAsmParser
[ 85%] Built target LLVMSparcDesc
[ 85%] Built target LLVMSparcDisassembler
[ 85%] Built target LLVMSystemZAsmParser
[ 85%] Built target LLVMSystemZAsmPrinter
[ 85%] Built target LLVMSystemZDisassembler
[ 85%] Built target LLVMSystemZInfo
[ 86%] Built target LLVMMipsCodeGen
[ 89%] Built target LLVMSystemZDesc
[ 89%] Built target LLVMSparcCodeGen
[ 89%] Built target LLVMX86AsmParser
[ 89%] Built target LLVMX86Disassembler
[ 90%] Built target LLVMX86AsmPrinter
[ 91%] Built target LLVMPowerPCCodeGen
[ 91%] Built target LLVMX86Utils
[ 91%] Built target LLVMX86Info
[ 91%] Built target LLVMXCoreDisassembler
[ 93%] Built target LLVMXCoreAsmPrinter
[ 93%] Built target LLVMXCoreInfo
[ 93%] Built target LLVMX86DC
[ 93%] Built target LLVMSystemZCodeGen
[ 93%] Built target LLVMX86Desc
[ 93%] Built target llvm-profdata
[ 93%] Built target LLVMXCoreDesc
[ 93%] Built target lli-child-target
[ 93%] Built target llvm-as
[ 93%] Built target llvm-cat
[ 93%] Built target llvm-extract
[ 93%] Built target llvm-link
[ 93%] Built target llvm-split
[ 93%] Built target llvm-cov
[ 93%] Built target llvm-modextract
[ 93%] Built target LLVMXCoreCodeGen
[ 93%] Built target bugpoint
[ 93%] Built target llvm-stress
[ 94%] Built target LLVMAMDGPUAsmPrinter
[ 94%] Built target verify-uselistorder
[ 94%] Built target BugpointPasses
[ 94%] Built target LLVMAMDGPUAsmParser
[ 94%] Built target LLVMAMDGPUDisassembler
[ 94%] Built target llvm-nm
[ 94%] Built target llvm-mc
[ 94%] Built target sancov
[ 94%] Built target llvm-mccfg
[ 94%] Built target llvm-objdump
[ 95%] Built target LLVMX86CodeGen
[ 97%] Built target llvm-cxxdump
[ 97%] Built target llvm-lto
[ 97%] Built target llvm-ar
[ 97%] Built target llc
[ 97%] Built target LTO
[ 97%] Built target llvm-dsymutil
[ 97%] Built target lli
[ 97%] Built target llvm-c-test
[ 97%] Built target llvm-dc
[ 97%] Built target llvm-dec
[ 98%] Built target llvm-rtdyld
[ 98%] Linking CXX executable ../../bin/llvm-dwp
[ 98%] Built target llvm-ranlib
[ 98%] Built target llvm-lto2
[100%] Built target llvm-lib
[100%] Built target llvm-xray
[100%] Linking CXX executable ../../bin/opt
[100%] Built target llvm-dwp
[100%] Built target opt
m-zakeri commented 5 years ago

I have this problem when building llvm in visual studio in 2019.

image