Set the Makevars file in highest piority for R.

[huyiqun]

Trying to install multi-thread version in R on OS X. Following the instruction here. Have done with the OpemMP compiler but couldn't understand what it means/how to "set the Makevars file in highest priority for R".

[ddydng]

Same here. Don't have a clue. @huyiqun did you finally get R multithread version working?

[thirdwing]

It simply means you create a folder ~/.R and put a Makevars in it.

[ddydng]

Hi @thirdwing I do not know how to write a Makevars so I copied the Makevars inside xgboost/R-package/src to ~/.R and I get following errors:

xgboost_R.cc:2:10: fatal error: 'dmlc/logging.h' file not found
#include <dmlc/logging.h>
         ^
1 error generated.
make: *** [xgboost_R.o] Error 1
ERROR: compilation failed for package ‘xgboost’
* removing ‘/Library/Frameworks/R.framework/Versions/3.2/Resources/library/xgboost’

After looking at the Makevars I think it is written to be used in that specified original location xgboost/R-package/src. So the question is, how to write this Makevars? This Makevars has to make these commands work in R, provided in instructions:

install.packages("drat", repos="https://cran.rstudio.com")
drat:::addRepo("dmlc")
install.packages("xgboost", repos="http://dmlc.ml/drat/", type = "source")

Another thing: the code above does not seem to have anything to do with the contents in xgboost/R-package. It is a bit weird to me since in that directory there is Makevars and everything else needed to compile the library. It seems the original purpose of that folder is to let users install the library from it. Is there a way to directly install it from that directory?

[jimthompson5802]

@choushishi This is what I have in my ~/.R/Makevars

CC=clang-omp
#CC=clang
CXX=clang-omp++
#CXX=clang++
SHLIB_OPENMP_CFLAGS= -fopenmp
SHLIB_OPENMP_CXXFLAGS= -fopenmp
SHLIB_OPENMP_FCFLAGS= -fopenmp
SHLIB_OPENMP_FFLAGS= -fopenmp

[ddydng]

@jimthompson5802

Thanks for sharing the code, however it does not seem working after I created the ~/.R/Makevars and put that code in. I run brew install clang-omp before running the installation command but I get following errors:

* installing *source* package ‘xgboost’ ...
** libs
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc-core/include -I./rabit/include -I. -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=0 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include   -fopenmp  -fPIC  -Wall -mtune=core2 -g -O2 -c xgboost_R.cc -o xgboost_R.o
In file included from xgboost_R.cc:3:
./dmlc-core/include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will be compiled into single-thread code. Use OpenMP-enabled compiler to get benefit of multi-threading. [-W#pragma-messages]
#pragma message("Warning: OpenMP is not available, "                    \
        ^
1 warning generated.
clang++ -std=c++11 -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc-core/include -I./rabit/include -I. -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=0 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include   -fopenmp  -fPIC  -Wall -mtune=core2 -g -O2 -c xgboost_custom.cc -o xgboost_custom.o
In file included from xgboost_custom.cc:5:
In file included from ./include/xgboost/logging.h:13:
In file included from ./include/xgboost/./base.h:10:
./dmlc-core/include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will be compiled into single-thread code. Use OpenMP-enabled compiler to get benefit of multi-threading. [-W#pragma-messages]
#pragma message("Warning: OpenMP is not available, "                    \
        ^
1 warning generated.
clang-omp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I./include -I./dmlc-core/include -I./rabit/include -I. -DXGBOOST_STRICT_R_MODE=1 -DDMLC_LOG_BEFORE_THROW=0 -DDMLC_ENABLE_STD_THREAD=0 -DDMLC_DISABLE_STDIN=1 -DDMLC_LOG_CUSTOMIZE=1 -DXGBOOST_CUSTOMIZE_LOGGER=1 -DRABIT_CUSTOMIZE_MSG_ -DRABIT_STRICT_CXX98_ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c xgboost_assert.c -o xgboost_assert.o
xgboost_assert.c:2:10: fatal error: 'stdio.h' file not found
#include <stdio.h>
         ^
1 error generated.
make: *** [xgboost_assert.o] Error 1
ERROR: compilation failed for package ‘xgboost’
* removing ‘/Library/Frameworks/R.framework/Versions/3.2/Resources/library/xgboost’

I can see that ~/.R/Makevars is working because when building xgboost_assert.o the clang-omp is used. However when building the prior two objects maybe ~/.R/Makevars is somehow ignored.

[ddydng]

I finally get this to work by using following code modified from @jimthompson5802 's answer. The code is put into ~/.R/Makevars:

CC=gcc-5
CXX=g++-5
CXX1X = g++-5

SHLIB_OPENMP_CFLAGS= -fopenmp
SHLIB_OPENMP_CXXFLAGS= -fopenmp
SHLIB_OPENMP_FCFLAGS= -fopenmp
SHLIB_OPENMP_FFLAGS= -fopenmp

In my case R would search /Library/Frameworks/R.framework/Resources/etc/Makeconf for necessary parameters undefined in ~/.R/Makevars. It would try to find the setting for CXX1X in Makeconf. So that is where the clang++ complier come from in the errors in my previous post.

I get a bunch of [-Wunused-variable] warnings but I guess it's fine. No OpenMP is not available warning this time.

Also since I have followed the official building instructions for mac os, I have gcc installed and with it I successfully built the "main" xgboost with OpenMP before. Thus I suggets for anyone who have followed the same instructions like me have their Makevars set up like this.

[ben519]

@choushishi Thanks. Your solution seems to work for me, and I was really struggling to get the multi-thread version of xgboost installed. Although, I also get a bunch of [-Wunused-variable] warnings...

[ahmedahmedov]

The whole installation instruction is incomprehensible. I have followed the below steps:

1. git clone --recursive https://github.com/dmlc/xgboost -> Done
2. cd xgboost; cp make/config.mk ./config.mk; make -j4 -> Done (I have gcc++-6 installed with brew install gcc --without-multilib)
3. "Set the Makevars file in highest priority for R." -> What the heck is it supposed to mean? So I followed the instructions in this thread and created the Makevars file in ~/.R directory
4. Now, here it gets tricky. The instructions instruct us to run the following 3 lines of code inside R. install.packages("drat", repos="https://cran.rstudio.com") drat:::addRepo("dmlc") install.packages("xgboost", repos="http://dmlc.ml/drat/", type = "source") Fine. But, what about the compiled xgboost package inside my GitHub folder created in steps 1 & 2? If I install the package thru the repository source code, why would I compile it myself? That is, how do I link the compiled R package to install.packages?

[ahmedahmedov]

To make things worse, if you save yourself the trouble and simply install by typing: install.packages("xgboost"), it will actually install an older version.

[thirdwing]

I am sorry I don't where you find the instructions.

But step 4 is to install the pre-built pkg host on github.

[amaa11]

@choushishi. I use Rstudio on windows. I use Rcpp. Also I want use omp. But it does not work. When I compiler it, it ignore all the pragmas lines. I do not know ho to fix it.. Any solution?? Regards

[hcho3]

@amaa11 You should install Rtools, and CRAN installation should support OpenMP out of the box