LLVM 3.5 auto installation

[acornagl]

I would like to install the LLVM-Tracer tool and, if it is possible, I also would like that cmake installs LLVM3.5.

For this reason I performed the following commands:

$ mkdir llvm3.5Auto
$ git clone https://github.com/ysshao/LLVM-Tracer
$ cd LLVM-Tracer
$ mkdir build                                    
$ cd build
$ cmake .. -DLLVM_ROOT=/local_disk/cornagli/llvmTracer/llvm3.5Auto -DLLVM_RECOMMEND_VERSION="3.5" -DAUTOINSTALL=TRUE

At this point I obtained the following error:

-- The CXX compiler identification is GNU 5.4.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- use LLVM_ROOT : /local_disk/cornagli/llvmTracer/llvm3.5Auto
-- find LLVM-Config : /usr/bin/llvm-config
CMake Error at cmake-scripts/AutoInstaller/installLLVM.cmake:39 (if):
  if given arguments:

    "VERSION_LESS" "1.22"

  Unknown arguments specified
Call Stack (most recent call first):
  cmake-scripts/AutoInstaller/installLLVM.cmake:95 (find_xz)
  cmake-scripts/AutoInstaller/installLLVM.cmake:261 (configure_autoinstall)
  cmake-scripts/LLVMFinder/findLLVM.cmake:63 (autoinstall_llvm)
  cmake-scripts/LLVMFinder/findLLVM.cmake:133 (CHECK_LLVM_CONFIG_VERSION_OR_AUTOINSTALL)
  cmake-scripts/findAndSetLLVM.cmake:13 (FIND_LLVM)
  CMakeLists.txt:8 (include)

-- Configuring incomplete, errors occurred!
See also "/local_disk/cornagli/llvmTracer/LLVM-Tracer/build/CMakeFiles/CMakeOutput.log".

Is it possible to request to auto install LLVM version 3.5? Thanks.

[acornagl]

I commented the following lines (38 - 43) in LLVM-Trace/cmake-scripts/AutoInstaller and the installation process started:

 #tar version newer than 1.22 has support to xz compression
 if(${TAR_VERSION} VERSION_LESS 1.22)
  message(FATAL_ERROR "tar version not less than 1.22 : ${TAR_VERSION}")
 else()
   message(STATUS "tar version : ${TAR_VERSION}")
 endif()

The tar version that I'm using is: tar (GNU tar) 1.28

[rgly]

Although LLVM-Tracer only supports LLVM version of 3.4/3.5, the LLVM installer supports from 3.1 to 3.6.

CMake Error at cmake-scripts/AutoInstaller/installLLVM.cmake:39 (if):
  if given arguments:

    "VERSION_LESS" "1.22"

It shows that your cmake does not support VERSION_LESS in if-clauses. Either change VERSION_LESS to LESS, or commenting these statement may work. Skipping tar version check is fine if the tar command does support xz format.

Can you provide your cmake version? Using cmake --version. So that I can reproduce the bug.

Thanks

[acornagl]

Sure, my cmake version is: cmake version 3.5.1.

Anyways, after that I commented the previous lines, the cmake command fails with the following description:

[ 93%] Building CXX object tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/ChainedIncludesSource.cpp.o
In file included from /local_disk/cornagli/llvmTracer/LLVM-Tracer/build/autoinstaller/llvm-source/llvm/tools/clang/include/clang/Basic/TargetInfo.h:24:0,
                 from /local_disk/cornagli/llvmTracer/LLVM-Tracer/build/autoinstaller/llvm-source/llvm/tools/clang/lib/Frontend/ChainedIncludesSource.cpp:15:
/local_disk/cornagli/llvmTracer/LLVM-Tracer/build/autoinstaller/llvm-source/llvm/include/llvm/ADT/IntrusiveRefCntPtr.h: In instantiation of ‘llvm::IntrusiveRefCntPtr<T>::IntrusiveRefCntPtr(llvm::IntrusiveRefCntPtr<X>&&) [with X = {anonymous}::ChainedIncludesSource; T = clang::ExternalSemaSource]’:
/local_disk/cornagli/llvmTracer/LLVM-Tracer/build/autoinstaller/llvm-source/llvm/tools/clang/lib/Frontend/ChainedIncludesSource.cpp:211:10:   required from here
/local_disk/cornagli/llvmTracer/LLVM-Tracer/build/autoinstaller/llvm-source/llvm/include/llvm/ADT/IntrusiveRefCntPtr.h:137:8: error: ‘{anonymous}::ChainedIncludesSource* llvm::IntrusiveRefCntPtr<{anonymous}::ChainedIncludesSource>::Obj’ is private
     T* Obj;
        ^
/local_disk/cornagli/llvmTracer/LLVM-Tracer/build/autoinstaller/llvm-source/llvm/include/llvm/ADT/IntrusiveRefCntPtr.h:158:13: error: within this context
       S.Obj = 0;
             ^
tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/build.make:182: recipe for target 'tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/ChainedIncludesSource.cpp.o' failed
make[2]: *** [tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/ChainedIncludesSource.cpp.o] Error 1
CMakeFiles/Makefile2:17539: recipe for target 'tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/all' failed
make[1]: *** [tools/clang/lib/Frontend/CMakeFiles/clangFrontend.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2
CMake Error at cmake-scripts/AutoInstaller/installLLVM.cmake:245 (message):
  wrong version of LLVM

  This is a bug.  Please contact developers.
Call Stack (most recent call first):
  cmake-scripts/AutoInstaller/installLLVM.cmake:279 (check_llvm_version)
  cmake-scripts/LLVMFinder/findLLVM.cmake:63 (autoinstall_llvm)
  cmake-scripts/LLVMFinder/findLLVM.cmake:133 (CHECK_LLVM_CONFIG_VERSION_OR_AUTOINSTALL)
  cmake-scripts/findAndSetLLVM.cmake:13 (FIND_LLVM)
  CMakeLists.txt:8 (include)

-- Configuring incomplete, errors occurred!
See also "/local_disk/cornagli/llvmTracer/LLVM-Tracer/build/CMakeFiles/CMakeOutput.log".

I attach to you the CMakeOutput.log file (I changed the extension to .txt, otherwise I can not attach it) CMakeOutput.txt .

Thanks!

[rgly]

I guess that LLVM 3.5 is not compatible with GCC 5.4. At the time LLVM 3.5 released, GCC 5 was not here yet.

Newer compiler may use stricter rules to check C++ syntax. I recommend you to use GCC 4.7/4.8/4.9. At least I tested these scripts using GCC 4.8.

[acornagl]

I see, thanks to your suggestion now the cmake terminate succesfully (I updated the command to: cmake .. -DLLVM_ROOT=/local_disk/cornagli/llvmTracer/llvm3.5Auto -DLLVM_RECOMMEND_VERSION="3.5" -DAUTOINSTALL=TRUE -DCMAKE_CXX_COMPILER:FILEPATH=/usr/bin/g++-4.8).

Unfortunately, I still have some problems with the make install command. I have set the the following variables:

$ export PATH=/local_disk/cornagli/llvmTracer/llvm3.5Auto/bin:$PATH      
$ LLVM_SO=/local_disk/cornagli/llvmTracer/llvm3.5Auto/lib 
$ export LLVM_SO 

The result of the make install is:

$ make install                                                     
[  7%] Building CXX object full-trace/CMakeFiles/full_trace.dir/full_trace.cpp.o
/local_disk/cornagli/llvmTracer/LLVM-Tracer/full-trace/full_trace.cpp:10:28: fatal error: llvm/DebugInfo.h: No such file or directory
 #include "llvm/DebugInfo.h"
                            ^
compilation terminated.
full-trace/CMakeFiles/full_trace.dir/build.make:62: recipe for target 'full-trace/CMakeFiles/full_trace.dir/full_trace.cpp.o' failed
make[2]: *** [full-trace/CMakeFiles/full_trace.dir/full_trace.cpp.o] Error 1
CMakeFiles/Makefile2:85: recipe for target 'full-trace/CMakeFiles/full_trace.dir/all' failed
make[1]: *** [full-trace/CMakeFiles/full_trace.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2

If I comment the line number 10 (the include statement) in full_trace.cpp, I obtain the following errors:

/local_disk/cornagli/llvmTracer/LLVM-Tracer/ast-pass/GetLabeledStmts.cpp: In function ‘int main(int, const char**)’:
/local_disk/cornagli/llvmTracer/LLVM-Tracer/ast-pass/GetLabeledStmts.cpp:158:36: error: no matching function for call to ‘clang::tooling::CommonOptionsParser::CommonOptionsParser(int&, const char**&)’
   CommonOptionsParser op(argc, argv);
                                    ^
In file included from /local_disk/cornagli/llvmTracer/LLVM-Tracer/ast-pass/GetLabeledStmts.cpp:32:0:
/local_disk/cornagli/llvmTracer/llvm3.5Auto/include/clang/Tooling/CommonOptionsParser.h:73:3: note: candidate: clang::tooling::CommonOptionsParser::CommonOptionsParser(int&, const char**, llvm::cl::OptionCategory&, const char*)
   CommonOptionsParser(int &argc, const char **argv,
   ^
/local_disk/cornagli/llvmTracer/llvm3.5Auto/include/clang/Tooling/CommonOptionsParser.h:73:3: note:   candidate expects 4 arguments, 2 provided
/local_disk/cornagli/llvmTracer/llvm3.5Auto/include/clang/Tooling/CommonOptionsParser.h:63:7: note: candidate: clang::tooling::CommonOptionsParser::CommonOptionsParser(clang::tooling::CommonOptionsParser&&)
 class CommonOptionsParser {
       ^
/local_disk/cornagli/llvmTracer/llvm3.5Auto/include/clang/Tooling/CommonOptionsParser.h:63:7: note:   candidate expects 1 argument, 2 provided
/local_disk/cornagli/llvmTracer/LLVM-Tracer/ast-pass/GetLabeledStmts.cpp:161:78: error: no matching function for call to ‘clang::tooling::ClangTool::run(std::unique_ptr<clang::tooling::FrontendActionFactory>)’
   int result = Tool.run(newFrontendActionFactory<LabeledStmtFrontendAction>());
                                                                              ^
In file included from /local_disk/cornagli/llvmTracer/LLVM-Tracer/ast-pass/GetLabeledStmts.cpp:33:0:
/local_disk/cornagli/llvmTracer/llvm3.5Auto/include/clang/Tooling/Tooling.h:283:7: note: candidate: int clang::tooling::ClangTool::run(clang::tooling::ToolAction*)
   int run(ToolAction *Action);
       ^
/local_disk/cornagli/llvmTracer/llvm3.5Auto/include/clang/Tooling/Tooling.h:283:7: note:   no known conversion for argument 1 from ‘std::unique_ptr<clang::tooling::FrontendActionFactory>’ to ‘clang::tooling::ToolAction*’

[rgly]

Recent updates broke the compatibility with LLVM 3.5. I made two git format patches to fix them, and will send merge request later, because I need to clean up experimental commits from my fork of LLVM-Tracer.

Here is the content of patches, patch files is provided at the bottom if you are in hurry.

--- a/ast-pass/GetLabeledStmts.cpp
+++ b/ast-pass/GetLabeledStmts.cpp
@@ -37,6 +37,12 @@ using namespace clang::driver;
 using namespace clang::tooling;
 using namespace llvm;

+// The option categories, to group options in the man page. It is useless
+// here, since we don't have any new options in this tool.
+// Declaring here because CommonOptionsParser requires this one in LLVM 3.5.
+// http://llvm.org/docs/CommandLine.html#grouping-options-into-categories
+cl::OptionCategory GetLabelStmtsCat("GetLabelStmts options");
+
 // Maps pairs of (func_name, label_name) to line numbers.
 static std::map<std::pair<std::string, std::string>, unsigned> labelMap;
 static const std::string outputFileName = "labelmap";
@@ -155,9 +161,21 @@ static void cleanup() {
 }

 int main(int argc, const char** argv) {
+#if (LLVM_VERSION == 34)
   CommonOptionsParser op(argc, argv);
+#elif (LLVM_VERSION == 35)
+  CommonOptionsParser op(argc, argv, GetLabelStmtsCat);
+#endif
+
   ClangTool Tool(op.getCompilations(), op.getSourcePathList());
   cleanup();
-  int result = Tool.run(newFrontendActionFactory<LabeledStmtFrontendAction>());
+
+  // In llvm 3.4, newFrontendActionFactory returns raw pointers.
+  // In llvm 3.5, it returns unique_ptr<>
+  // Use unique_ptr to keep pointers, therefore being compatible with
+  // LLVM 3.4/3.5
+  std::unique_ptr<FrontendActionFactory>
+        actionfactory(newFrontendActionFactory<LabeledStmtFrontendAction>());
+  int result = Tool.run(actionfactory.get());
   return result;
 }
diff --git a/full-trace/full_trace.cpp b/full-trace/full_trace.cpp
index ab13c7c..485e856 100644
--- a/full-trace/full_trace.cpp
+++ b/full-trace/full_trace.cpp
@@ -7,7 +7,6 @@
 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/Metadata.h"
 #include "llvm/IR/Type.h"
-#include "llvm/DebugInfo.h"
 #include "llvm/Support/CommandLine.h"
 #include <cstring>
 #include <cstdlib>
@@ -20,6 +19,12 @@
 #include "SlotTracker.h"
 #include "full_trace.h"

+#if (LLVM_VERSION == 34)
+  #include "llvm/DebugInfo.h"
+#elif (LLVM_VERSION == 35)
+  #include "llvm/IR/DebugInfo.h"
+#endif
+
 #define RESULT_LINE 19134
 #define FORWARD_LINE 24601
 #define DMA_FENCE 97

Since Github reject patch suffix to attached files, I renamed them as txt files. Please change the suffix as patch, then git am < file.patch to your git repo.

0001-bugfix-llvm-DebugInfo.h-broke-LLVM-3.5-compatiabilit.txt 0002-make-GetLabeledStmts-compatiable-with-LLVM-3.5.txt

[xyzsam]

Thanks @rgly!

[acornagl]

Thank you very much for the patches @rgly !

Unfortunately, I tried them but I encountered a different problem:

[ 50%] Linking CXX executable get-labeled-stmts
CMakeFiles/get-labeled-stmts.dir/GetLabeledStmts.cpp.o: In function `main':
GetLabeledStmts.cpp:(.text.startup+0x69): undefined reference to `clang::tooling::ClangTool::ClangTool(clang::tooling::CompilationDatabase const&, llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >)'
collect2: error: ld returned 1 exit status
ast-pass/CMakeFiles/get-labeled-stmts.dir/build.make:124: recipe for target 'ast-pass/get-labeled-stmts' failed
make[2]: *** [ast-pass/get-labeled-stmts] Error 1
CMakeFiles/Makefile2:228: recipe for target 'ast-pass/CMakeFiles/get-labeled-stmts.dir/all' failed
make[1]: *** [ast-pass/CMakeFiles/get-labeled-stmts.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2

[rgly]

I can not reproduce your issue. Needs more information about your system.

1. What is your Linux distribution?
2. How many compilers on your system?
3. Did you build LLVM-Tracer with the same compiler building LLVM-3.5?

Here is my system information:

CPU : Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz Kernel : 3.13.0-37-generic OS : Linux Mint 17.1 Rebecca Shell : GNU bash 4.3.11 g++ 4.8.5 (Ubuntu 4.8.5-2ubuntu1~14.04.1) cmake 3.6.2

Here is my log, I didn't set any environment variable. Build llvm-3.5 & LLVM-Traver from scratch via CMake.

rgly@rgly-Linux-Mint ~/Source/aladdin/build $ cmake ../LLVM-Tracer/ -DLLVM_ROOT=$HOME/lib/llvm-3.5 -DLLVM_RECOMMEND_VERSION="3.5" -DAUTOINSTALL=TRUE
   .....  delete LLVM build messages  ........
-- Installing: /home/rgly/lib/llvm-3.5/share/llvm/cmake/./ChooseMSVCCRT.cmake
-- Installing: /home/rgly/lib/llvm-3.5/share/llvm/cmake/./FindSphinx.cmake
-- Installing: /home/rgly/lib/llvm-3.5/share/llvm/cmake/./AddLLVMDefinitions.cmake
-- Installing: /home/rgly/lib/llvm-3.5/share/llvm/cmake/./AddLLVM.cmake
-- Installing: /home/rgly/lib/llvm-3.5/share/llvm/cmake/./HandleLLVMStdlib.cmake
-- Installing: /home/rgly/lib/llvm-3.5/share/llvm/cmake/./AddSphinxTarget.cmake
-- Finished installing LLVM.
-- find LLVM-Config : /home/rgly/lib/llvm-3.5/bin/llvm-config
-- LLVM version : 3.5.2svn
-- LLVM_LINK found : /home/rgly/lib/llvm-3.5/bin/llvm-link
-- LLVM_OPT found : /home/rgly/lib/llvm-3.5/bin/opt
-- LLVM_LLC found : /home/rgly/lib/llvm-3.5/bin/llc
-- LLVM_COMPILER found : /home/rgly/lib/llvm-3.5/bin/clang-3.5
-- Found Curses: /usr/lib/x86_64-linux-gnu/libcurses.so  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rgly/Source/aladdin/build
rgly@rgly-Linux-Mint ~/Source/aladdin/build $ make 
Scanning dependencies of target full_trace
[  7%] Building CXX object full-trace/CMakeFiles/full_trace.dir/full_trace.cpp.o
In file included from /home/rgly/Source/aladdin/LLVM-Tracer/full-trace/full_trace.cpp:19:0:
/home/rgly/Source/aladdin/LLVM-Tracer/full-trace/SlotTracker.h:137:21: warning: ‘SlotTracker* createSlotTracker(const llvm::Value*)’ defined but not used [-Wunused-function]
 static SlotTracker *createSlotTracker(const Value *V) {
                     ^
[ 14%] Linking CXX shared library full_trace.so
[ 14%] Built target full_trace
Scanning dependencies of target trace_logger
[ 21%] Generating trace_logger.obj.llvm
[ 28%] Generating trace_logger.unopt.llvm
[ 35%] Generating trace_logger.llvm
[ 35%] Built target trace_logger
Scanning dependencies of target PROFILE_FUNC
[ 35%] Built target PROFILE_FUNC
Scanning dependencies of target get-labeled-stmts
[ 42%] Building CXX object ast-pass/CMakeFiles/get-labeled-stmts.dir/GetLabeledStmts.cpp.o
In file included from /home/rgly/lib/llvm-3.5/include/clang/AST/Decl.h:17:0,
                 from /home/rgly/lib/llvm-3.5/include/clang/AST/ASTTypeTraits.h:20,
                 from /home/rgly/lib/llvm-3.5/include/clang/AST/ASTContext.h:18,
                 from /home/rgly/lib/llvm-3.5/include/clang/AST/AST.h:18,
                 from /home/rgly/Source/aladdin/LLVM-Tracer/ast-pass/GetLabeledStmts.cpp:24:
/home/rgly/lib/llvm-3.5/include/clang/AST/APValue.h: In member function ‘clang::APValue::APSInt& clang::APValue::getInt()’:
/home/rgly/lib/llvm-3.5/include/clang/AST/APValue.h:202:34: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
     return *(APSInt*)(char*)Data.buffer;
                                  ^
....... delete the same warnings ........
                                             ^
/home/rgly/lib/llvm-3.5/include/clang/AST/APValue.h:389:45: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
     ((AddrLabelDiffData*)(char*)Data.buffer)->RHSExpr = RHSExpr;
                                             ^
[ 50%] Linking CXX executable get-labeled-stmts
[ 50%] Built target get-labeled-stmts
Scanning dependencies of target Tracer_triad
[ 57%] Generating triad.obj.llvm
[ 64%] Generating triad.unopt.llvm
[ 71%] Generating triad.llvm
[ 78%] Generating triad-opt.llvm
[ 85%] Generating full.llvm
[ 92%] Generating full.s
[100%] Generating triad-instrumented
[100%] Built target Tracer_triadt

And get-labeled-stmts works well.

rgly@rgly-Linux-Mint ~/Source/aladdin/build/ast-pass $ ./get-labeled-stmts ../../LLVM-Tracer/ast-pass/test.c -- -I$$HOME/lib/llvm-3.5/lib/clang/3.4/include -I/usr/include/c++/4.8 -I/usr/include/x86_64-linux-gnu/c++/4.8 -I/usr/include/x86_64-linux-gnu -I/usr/include -I/usr/include/c++/4.8/backward -I/usr/lib/gcc/x86_64-linux-gnu/4.8/include
rgly@rgly-Linux-Mint ~/Source/aladdin/build/ast-pass $ cat labelmap 
bar/macro_label 15
different_line/label_here 40
different_line/label_in_between 40
foo/foo_label 20
nested/inner 26
nested/outer 25

[acornagl]

1. What is your Linux distribution?

Ubuntu 16.04-LTS

How many compilers on your system?

More than one:

1. g++-5 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609, and
2. g++-4.8 (Ubuntu 4.8.5-4ubuntu2) 4.8.5.

For this reason I set -DCMAKE_CXX_COMPILER:FILEPATH=/usr/bin/g++-4.8 in the cmake command.

Did you build LLVM-Tracer with the same compiler building LLVM-3.5?

I compiled LLVM-3.5 via the cmake script (DAUTOINSTALL=TRUE), so I would expect that both of them have been compiled with g++-4.8. Is this assumption correct?

[acornagl]

Thanks, finally I have been able both to compile LLVM-Tracer and to run the example.

I changed the occurrences of g++ in g++-4.8 in the following two file (it seems that the -DCMAKE_CXX_COMPILER:FILEPATH=/usr/bin/g++-4.8 option had not effects for them):

1. build/ast-pass/CMakeFiles/get-labeled-stmts.dir/build.make, and
2. build/ast-pass/CMakeFiles/get-labeled-stmts.dir/link.txt.

After the modifications, and with the proposed git patches applied, I have been able to reproduce the results for get-labeled-stmts.

From my point of view, the issue can be considered as resolved. Thanks.