search

RGF-team / rgf

Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.
378 stars 58 forks source link

document building FastRGF with Clang and AppleClang #319

Closed StrikerRUS closed 4 years ago

StrikerRUS commented 4 years ago

Follow-up #316.

StrikerRUS commented 4 years ago

Hmmm, seems that compiled with Clang and AppleClang FastRGF binaries cannot pass our simple test during rgf_python installation. https://github.com/RGF-team/rgf/blob/49756aedb39cd9a6908ede77cd8e0566a061bd84/python-package/setup.py#L116-L153 Refer to the corresponding Travis jobs (with debug logs): https://travis-ci.org/RGF-team/rgf/builds/657496643

Although FastRGF compiled successfully

Clang:

  INFO:rgf_python:Starting to compile FastRGF executable files.
  cmake version 3.12.4
  CMake suite maintained and supported by Kitware (kitware.com/cmake).
INFO:rgf_python:Trying to build executable files with CMake.
  -- The C compiler identification is Clang 7.0.0
  -- The CXX compiler identification is Clang 7.0.0
  -- Check for working C compiler: /usr/local/clang-7.0.0/bin/clang
  -- Check for working C compiler: /usr/local/clang-7.0.0/bin/clang -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /usr/local/clang-7.0.0/bin/clang++
  -- Check for working CXX compiler: /usr/local/clang-7.0.0/bin/clang++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  Use OpenMP for multithreading
  -- Found OpenMP_C: -fopenmp=libomp (found version "3.1")
  -- Found OpenMP_CXX: -fopenmp=libomp (found version "3.1")
  -- Found OpenMP: TRUE (found version "3.1")
  C++ compiler: /usr/local/clang-7.0.0/bin/clang++
  C++ options: -O3 -std=c++11 -fopenmp=libomp -pthread -ftree-vectorize -ffast-math
  C++ definitions: USE_OMP
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-req-build-qiln_j36/compile/FastRGF/build  829
  Scanning dependencies of target base
    [  5%] Building CXX object src/base/CMakeFiles/base.dir/utils.cpp.o
    [ 10%] Building CXX object src/base/CMakeFiles/base.dir/data.cpp.o
    [ 15%] Building CXX object src/base/CMakeFiles/base.dir/discretization.cpp.o
    [ 21%] Building CXX object src/base/CMakeFiles/base.dir/classifier.cpp.o
    [ 26%] Linking CXX static library libbase.a
    [ 26%] Built target base
    Scanning dependencies of target forest
    [ 31%] Building CXX object src/forest/CMakeFiles/forest.dir/dtree_trainer.cpp.o
    [ 36%] Building CXX object src/forest/CMakeFiles/forest.dir/dtree.cpp.o
    [ 42%] Building CXX object src/forest/CMakeFiles/forest.dir/forest.cpp.o
    [ 47%] Linking CXX static library libforest.a
    [ 47%] Built target forest
    Scanning dependencies of target forest_train
    [ 52%] Building CXX object src/exe/CMakeFiles/forest_train.dir/forest_train.cpp.o
    [ 57%] Linking CXX executable forest_train
    [ 57%] Built target forest_train
    Scanning dependencies of target forest_predict
    [ 63%] Building CXX object src/exe/CMakeFiles/forest_predict.dir/forest_predict.cpp.o
    [ 68%] Linking CXX executable forest_predict
    [ 68%] Built target forest_predict
    Scanning dependencies of target discretized_gendata
    [ 73%] Building CXX object src/exe/CMakeFiles/discretized_gendata.dir/discretized_gendata.cpp.o
    [ 78%] Linking CXX executable discretized_gendata
    [ 78%] Built target discretized_gendata
    Scanning dependencies of target auc
    [ 84%] Building CXX object src/exe/CMakeFiles/auc.dir/auc.cpp.o
    [ 89%] Linking CXX executable auc
    [ 89%] Built target auc
    Scanning dependencies of target discretized_trainer
    [ 94%] Building CXX object src/exe/CMakeFiles/discretized_trainer.dir/discretized_trainer.cpp.o
    [100%] Linking CXX executable discretized_trainer
    [100%] Built target discretized_trainer
  [ 26%] Built target base
    [ 47%] Built target forest
    [ 57%] Built target forest_train
    [ 68%] Built target forest_predict
    [ 78%] Built target discretized_gendata
    [ 89%] Built target auc
    [100%] Built target discretized_trainer
    Install the project...
    -- Install configuration: ""
    -- Installing: /tmp/pip-req-build-qiln_j36/compile/FastRGF/bin/forest_train
    -- Installing: /tmp/pip-req-build-qiln_j36/compile/FastRGF/bin/forest_predict
  ERROR:rgf_python:Compilation of FastRGF executable files failed. Please build from binaries on your own and specify path to the compiled files in the config file.

AppleClang:

  INFO:rgf_python:Starting to compile FastRGF executable files.
  cmake version 3.16.2
  CMake suite maintained and supported by Kitware (kitware.com/cmake).
INFO:rgf_python:Trying to build executable files with CMake.
  -- The C compiler identification is AppleClang 11.0.0.11000033
  -- The CXX compiler identification is AppleClang 11.0.0.11000033
  -- Check for working C compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
  -- Check for working C compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Check for working CXX compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
  -- Check for working CXX compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  Use OpenMP for multithreading
  -- Found OpenMP_C: -Xclang -fopenmp (found version "3.1")
  -- Found OpenMP_CXX: -Xclang -fopenmp (found version "3.1")
  -- Found OpenMP: TRUE (found version "3.1")
  C++ compiler: /Applications/Xcode-11.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
  C++ options: -O3 -std=c++11 -Xclang -fopenmp -pthread -ftree-vectorize -ffast-math
  C++ definitions: USE_OMP
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /private/var/folders/17/5mc7816d3mndxjqgplq6057w0000gn/T/pip-req-build-o1fxjh9r/compile/FastRGF/build
  ERROR:rgf_python:Compilation of FastRGF executable files failed. Please build from binaries on your own and specify path to the compiled files in the config file.

I guess this is somehow connected to #92.

I also tried to decrease the lvl of compiler optimizations - no luck.

So I'm going to revert the changes related to removal of used compiler restrictions in rgf_python. I think for now we should stick to the current policy of allowing only g++-5 and later as only this compiler can pass CI tests.

cc @ankane

fukatani commented 4 years ago

Thank you for your investigation and PR. Revirting clang and apple clang will be done with other PR?

StrikerRUS commented 4 years ago

@fukatani Do you mean to revert #316? I think we can leave the possibility to compile with AppleClang and Clang as it compiles successfully now

Although FastRGF compiled successfully https://github.com/RGF-team/rgf/pull/319#issuecomment-594900317

and we have the evidence of successful run: https://github.com/RGF-team/rgf/pull/316#issuecomment-591304581. So, I guess Clang-compiled binaries work well under the some conditions (maybe with large enough datasets). It will be consistent with our previous policy to not prohibit compilation itself with g++-4, while those binaries cannot pass our tests.

fukatani commented 4 years ago

Ah, I see. Thank you for your contribution!