Open aadler opened 2 years ago
This does not work either:
D:\LightGBM\build>cmake -A x64 -DUSE_GPU=1 -DBOOST_ROOT=C:/local/boost_1_78_0 ..
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.31.31105.0
-- The CXX compiler identification is MSVC 19.31.31105.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/lib/x64/OpenCL.lib (found version "2.2")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/include
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake (found suitable version "1.78.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: D:/LightGBM/build
And adding -DBOOST_LIBRARYDIR=C:/local/boost_1_78_0/lib64-msvc-14.3
doesn't help either
Thanks for opening this issue @aadler, and for all the details!
install.libs.R:142:45: unexpected symbol
blegh sorry about that. I suspect that the argument parsing in that script is not handling Windows paths with :
correctly. I'm only aware of it having been tested on Linux systems (like in https://github.com/microsoft/LightGBM/pull/3779#issuecomment-762409057).
I'll put up a pull request shortly to fix that.
When I pass --msys2, the build works properly
This is very encouraging! I think then it the problem is limited to "[R-package] cannot build GPU version with MSVC and Rtools 4.x", right?
This does not work either
Thank you for trying the installation with just cmake
and not the R package. That's super helpful for debugging! build_r.R
is basically just used to put together exactly a cmake
command like that. Could you clarify what you mean by "doesn't work", like share the logs from make -j4
run after cmake ...
?
Thank you, @jameslamb, for the quick response.
What I mean by "doesn't work" is that I pass a boost root directory "C:/local/boost_1_78_0" that is not registered by the script: "found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake"
Run with what I believe should be the proper invocation, I get this:
D:\LightGBM\build>cmake -A x64 -DUSE_GPU=1 -DBOOST_ROOT=C:/local/boost_1_78_0 -DBOOST_LIBRARYDIR=C:/local/boost_1_78_0/lib64-msvc-14.3 ..
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.31.31105.0
-- The CXX compiler identification is MSVC 19.31.31105.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/lib/x64/OpenCL.lib (found version "2.2")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/include
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake (found suitable version "1.78.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_LIBRARYDIR
-- Build files have been written to: D:/LightGBM/build
Note that it does not find the boost binaries which I passed. Now running the actual build will return a gazillion and two errors:
D:\LightGBM\build>cmake --build . --target ALL_BUILD --config Release > LGBM.log
Will return a huge log file chock full of errors. The entire near 3K line logfile is here: LGBM.log
Start (Lines 1–33):
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
Checking Build System
Building Custom Rule D:/LightGBM/CMakeLists.txt
c_api.cpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(114,1): warning C4005: '_CRT_WIDE': macro redefinition [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(292): message : see previous definition of '_CRT_WIDE' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(155,1): warning C4005: '__CRTDECL': macro redefinition [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(399): message : see previous definition of '__CRTDECL' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(325,1): warning C4005: '_CRT_DEPRECATE_TEXT': macro redefinition [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(350): message : see previous definition of '_CRT_DEPRECATE_TEXT' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(366,51): error C2065: 'uintptr_t': undeclared identifier [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(366,61): error C2146: syntax error: missing ')' before identifier '_StackCookie' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(367,67): error C2065: 'uintptr_t': undeclared identifier [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(367,77): error C2146: syntax error: missing ')' before identifier '_StackCookie' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(371,35): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(371,18): error C2146: syntax error: missing ';' before identifier '__security_cookie' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\corecrt.h(80,53): error C2378: 'uintptr_t': redefinition; symbol cannot be overloaded with a typedef [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(371): message : see declaration of 'uintptr_t' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\stddef.h(26,28): error C2143: syntax error: missing ';' before '__cdecl' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\stddef.h(26,28): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\stddef.h(26,18): error C2370: 'uintptr_t': redefinition; different storage class [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(371): message : see declaration of 'uintptr_t' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\stddef.h(26,56): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\cstdint(52,13): error C2874: using-declaration causes a multiple declaration of 'uintptr_t' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\corecrt.h(80): message : see declaration of 'uintptr_t' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\cstdint(86,17): error C2874: using-declaration causes a multiple declaration of 'uintptr_t' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\corecrt.h(80): message : see declaration of 'uintptr_t' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\stdio.h(184,64): error C2144: syntax error: 'int' should be preceded by ';' [D:\LightGBM\build\lightgbm_capi_objs.vcxproj]
End (Lines 2899–2910):
C:\rtools42\x86_64-w64-mingw32.static.posix\include\stdio.h(1332,3): error C2447: '{': missing function header (old-style formal list?) (compiling source file D:\LightGBM\src\treelearner\linear_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\swprintf.inl(27,1): error C2062: type 'int' unexpected (compiling source file D:\LightGBM\src\treelearner\linear_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\swprintf.inl(28,1): error C2143: syntax error: missing ';' before '{' (compiling source file D:\LightGBM\src\treelearner\linear_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\swprintf.inl(28,1): error C2447: '{': missing function header (old-style formal list?) (compiling source file D:\LightGBM\src\treelearner\linear_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\swprintf.inl(34,1): error C2062: type 'int' unexpected (compiling source file D:\LightGBM\src\treelearner\linear_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\swprintf.inl(35,1): error C2143: syntax error: missing ';' before '{' (compiling source file D:\LightGBM\src\treelearner\linear_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\swprintf.inl(35,1): error C2447: '{': missing function header (old-style formal list?) (compiling source file D:\LightGBM\src\treelearner\linear_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\swprintf.inl(35,1): fatal error C1003: error count exceeds 100; stopping compilation (compiling source file D:\LightGBM\src\treelearner\linear_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\malloc.h(80,10): fatal error C1083: Cannot open include file: 'mm_malloc.h': No such file or directory (compiling source file D:\LightGBM\src\treelearner\tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\malloc.h(80,10): fatal error C1083: Cannot open include file: 'mm_malloc.h': No such file or directory (compiling source file D:\LightGBM\src\treelearner\gpu_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\malloc.h(80,10): fatal error C1083: Cannot open include file: 'mm_malloc.h': No such file or directory (compiling source file D:\LightGBM\src\treelearner\serial_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\malloc.h(80,10): fatal error C1083: Cannot open include file: 'mm_malloc.h': No such file or directory (compiling source file D:\LightGBM\src\treelearner\voting_parallel_tree_learner.cpp) [D:\LightGBM\build\lightgbm_objs.vcxproj]
Looking at another issue, I found that despite the script not registering -DBOOST_HOME etc, if I changed the name of C:\local\boost_1_78_0\lib64-msvc-14.3
to C:\local\boost_1_78_0\lib
and move the "local" subdirectories ahead of Rtolls42 in the PATH, they would be found.
This leads to another issue; I think they are related but would be happy to split out if needed. Using the advice given in https://github.com/microsoft/LightGBM/issues/5140, I checked out 3.3.2 and tried to build it. The CLI seems to complete, but I need the R package. Running Rscript build_r.R --use-gpu
shows that Visual Studio 17 2021 is not an acceptable option. I'm guessing this is a relatively simple fix eventually? The actual error I get is below. Thank you!
D:\LightGBM>Rscript build_r.R --use-gpu
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/external_libs/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_3.3.2.tar.gz'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_atomics.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_reduce.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/reduce_by_key_with_scan.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/iterator/detail/get_base_iterator_buffer.hpp'
* installing to library 'C:/R/RCurrent/R-4.2.0alpha/library'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/R/RCurrent/R-4.2.0alpha/library/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 16 2019'
-- Configuring incomplete, errors occurred!
CMake Error at CMakeLists.txt:32 (PROJECT):
See also "D:/Rtemp/RtmpoZLwxV/R.INSTALL12dc7fb76921/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
Generator
Visual Studio 16 2019
could not find any instance of Visual Studio.
Trying 'Visual Studio 15 2017'
CMake Error at CMakeLists.txt:32 (PROJECT):
-- Configuring incomplete, errors occurred!
Generator
Visual Studio 15 2017
could not find any instance of Visual Studio.
See also "D:/Rtemp/RtmpoZLwxV/R.INSTALL12dc7fb76921/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
Trying 'Visual Studio 14 2015'
CMake Error at CMakeLists.txt:32 (PROJECT):
-- Configuring incomplete, errors occurred!
Failed to run MSBuild command:
MSBuild.exe
to get the value of VCTargetsPath:
The system cannot find the file specified
See also "D:/Rtemp/RtmpoZLwxV/R.INSTALL12dc7fb76921/lightgbm/src/build/CMakeFiles/CMakeOutput.log".
Warning in eval(ei, envir) :
Building with Visual Studio failed. Attempting with MSYS2
CMake Error: Error: generator : MSYS Makefiles
Does not match the generator used previously: Visual Studio 14 2015
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
CMake Error: Error: generator : MSYS Makefiles
Does not match the generator used previously: Visual Studio 14 2015
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a different binary directory.
Error in .run_shell_command("cmake", c(cmake_args, "..")) :
Command failed with exit code: 1
* removing 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
* restoring previous 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
Error in .run_shell_command(install_cmd, install_args) :
Command failed with exit code: 1
Execution halted
Visual Studio 17 2022 support was added in #4889.
If you're building from source on v3.3.2
, you could try manually adding the following line to R-package/src/install.libs.R
.
Thanks. That worked for registering VS 17/2022, but we're back to the parsing issue. When nothing is passed other than --use-gpu
then despite finding the proper path when building the CLI interface, the R.build finds the wrong one. Please note the last lines in the quote below; the Rtools boost and the MM_ failures.
:\LightGBM>Rscript build_r.R --use-gpu
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/external_libs/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_3.3.2.tar.gz'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_atomics.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_reduce.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/reduce_by_key_with_scan.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/iterator/detail/get_base_iterator_buffer.hpp'
* installing to library 'C:/R/RCurrent/R-4.2.0alpha/library'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/R/RCurrent/R-4.2.0alpha/library/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 17 2022'
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.31.31105.0
-- The CXX compiler identification is MSVC 19.31.31105.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- R version passed into FindLibR.cmake: 4.2.0
-- Creating R.lib and R.def
Creating 'D:/Rtemp/RtmpGqfbKQ/R.INSTALL51ccda8528b/lightgbm/src/build/R.def' from 'C:/R/RCurrent/R-4.2.0alpha/bin/x64/R.dll'
Successfully created 'D:/Rtemp/RtmpGqfbKQ/R.INSTALL51ccda8528b/lightgbm/src/build/R.def'
-- Found LibR: C:/R/RCurrent/R-4.2.0alpha
-- LIBR_EXECUTABLE: C:/R/RCurrent/R-4.2.0alpha/bin/x64/R.exe
-- LIBR_INCLUDE_DIRS: C:/R/RCurrent/R-4.2.0alpha/include
-- LIBR_CORE_LIBRARY: C:/R/RCurrent/R-4.2.0alpha/bin/x64/R.dll
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/lib/x64/OpenCL.lib (found version "2.2")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.6/include
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Boost toolset is unknown (compiler MSVC 19.31.31105.0)
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake (found suitable version "1.78.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Rtemp/RtmpGqfbKQ/R.INSTALL51ccda8528b/lightgbm/src/build
This then leads to a failed build (excerpt below):
Successfully created build files for 'Visual Studio 17 2022'
Building lib_lightgbm
Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
Checking Build System
Building Custom Rule D:/Rtemp/RtmpGqfbKQ/R.INSTALL51ccda8528b/lightgbm/src/CMakeLists.txt
boosting.cpp
gbdt.cpp
gbdt_model_text.cpp
gbdt_prediction.cpp
prediction_early_stop.cpp
bin.cpp
config.cpp
config_auto.cpp
dataset.cpp
dataset_loader.cpp
file_io.cpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
json11.cpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(114,1): warning C4005: '_CRT_WIDE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(292): message : see previous definition of '_CRT_WIDE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(155,1): warning C4005: '__CRTDECL': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(399): message : see previous definition of '__CRTDECL' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(325,1): warning C4005: '_CRT_DEPRECATE_TEXT': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(350): message : see previous definition of '_CRT_DEPRECATE_TEXT' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(366,51): error C2065: 'uintptr_t': undeclared identifier (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(366,61): error C2146: syntax error: missing ')' before identifier '_StackCookie' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(367,67): error C2065: 'uintptr_t': undeclared identifier (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(367,77): error C2146: syntax error: missing ')' before identifier '_StackCookie' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(371,35): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(371,18): error C2146: syntax error: missing ';' before identifier '__security_cookie' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\boosting.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_model_text.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_prediction.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\prediction_early_stop.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\io\bin.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(114,1): warning C4005: '_CRT_WIDE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(292): message : see previous definition of '_CRT_WIDE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_model_text.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_model_text.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_prediction.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\gbdt_prediction.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\prediction_early_stop.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\boosting\prediction_early_stop.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\io\bin.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\src\io\bin.cpp) [D:\Rtemp\RtmpGqfbKQ\R.INSTALL51ccda8528b\lightgbm\src\build\_lightgbm.vcxproj]
It goes on and on.
When I try passing the boost directory, I get the original issue:
:\LightGBM>Rscript build_r.R --use-gpu --boost-librarydir=C:\local\boost_1_78_0\lib
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/external_libs/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_3.3.2.tar.gz'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_atomics.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_reduce.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/reduce_by_key_with_scan.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/iterator/detail/get_base_iterator_buffer.hpp'
* installing to library 'C:/R/RCurrent/R-4.2.0alpha/library'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/R/RCurrent/R-4.2.0alpha/library/00LOCK-lightgbm/00new/lightgbm
Error in source("install.libs.R", local = local.env) :
install.libs.R:141:45: unexpected symbol
140: # NOTE: build_r.R replaces the line below
141: command_line_args <- c("-DBOOST_LIBRARYDIR="C
^
* removing 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
* restoring previous 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
Error in .run_shell_command(install_cmd, install_args) :
Command failed with exit code: 1
Execution halted
Or
Error in source("install.libs.R", local = local.env) :
install.libs.R:141:70: unexpected string constant
140: # NOTE: build_r.R replaces the line below
141: command_line_args <- c("-DBOOST_LIBRARYDIR="/c/local/boost_1_78_0/lib""
^
Or
Error in source("install.libs.R", local = local.env) :
install.libs.R:141:45: unexpected symbol
140: # NOTE: build_r.R replaces the line below
141: command_line_args <- c("-DBOOST_LIBRARYDIR="c
^
* removing 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
* restoring previous 'C:/R/RCurrent/R-4.2.0alpha/library/lightgbm'
etc.
I appreciate your patience, @jameslamb. At this point, I'll stick with building the gpu-enabled version 3.3.2 from source using MSYS2, which works and can use the GPU (althogh for my problem, right now, it seems the CPU is faster :D ).
Thank you!!
Hi, @jameslamb. Just confirming this is still an issue with 3.3.2.99 version 9489f87. Thanks
Thanks for the nudge and sorry for the delay @aadler ! I also really appreciate "hey even as of the most recent commit this is still a bug" more than those "any updates?" types of comments people leave sometimes 😍
It's hard for me to test this (and possible fixes) because I don't have easy access to a Windows environment, but will try to get to it soon if no other maintainers do.
Hi @jameslamb. New version so I thought I'd check the BOOST issue. Still not working unless I am doing something wrong. Note where I say the libraries should be and note where CMake finds them :(
Thank you.
D:\LightGBM\build>cmake -A x64 -DUSE_GPU=1 -DBOOST_ROOT=D:/local/boost_1_80_0 -DBOOST_LIBRARYDIR=D:/local/boost_1_80_0/lib64-msvc-14.3 ..
-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.33.31629.0
-- The CXX compiler identification is MSVC 19.33.31629.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_3_0
-- Looking for CL_VERSION_3_0 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/lib/x64/OpenCL.lib (found version "3.0")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/include
-- Boost toolset is unknown (compiler MSVC 19.33.31629.0)
-- Boost toolset is unknown (compiler MSVC 19.33.31629.0)
-- Boost toolset is unknown (compiler MSVC 19.33.31629.0)
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.78.0/BoostConfig.cmake (found suitable version "1.78.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_LIBRARYDIR
-- Build files have been written to: D:/LightGBM/build
Hey @aadler could you try pulling the branch from #5607? It doesn't fix the "LightGBM uses RTools Boost instead of the one you asked for" issue yet, but I think it at least fixes the parsing of paths so the errors like "unexpected string constant"
or "unexpected symbol"
shouldn't happen any more.
You could pull it like this
git remote add upstream https://github.com/microsoft/LightGBM.git
git fetch upstream fix/r-gpu
git checkout fix/r-gpu
git submodule update --recursive
Hello. @jameslamb
I get the following error:
D:\LightGBM>git checkout fix/r-gpu
hint: If you meant to check out a remote tracking branch on, e.g. 'origin',
hint: you can do so by fully qualifying the name with the --track option:
hint:
hint: git checkout --track origin/<name>
hint:
hint: If you'd like to always have checkouts of an ambiguous <name> prefer
hint: one remote, e.g. the 'origin' remote, consider setting
hint: checkout.defaultRemote=origin in your config.
fatal: 'fix/r-gpu' matched multiple (2) remote tracking branches
Ok try
git checkout --track upstream/fix/r-gpu
True. None of those errors, but it is still finding the wrong Boost.
D:\LightGBM>Rscript build_r.R --use-gpu --boost-root=D:/local/boost_1_80_0/
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to build vignettes
-----------------------------------
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to D:/Rtemp/RtmpcFC2dN/Rinst765423f358f8/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 17 2022'
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.33.31629.0
-- The CXX compiler identification is MSVC 19.33.31629.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- R version passed into FindLibR.cmake: 4.2.2
-- Creating R.lib and R.def
Creating 'D:/Rtemp/RtmpcFC2dN/Rbuild765417941dd1/lightgbm/src/build/R.def' from 'C:/R/RCurrent/R-4.2.2patched/bin/x64/R.dll'
Successfully created 'D:/Rtemp/RtmpcFC2dN/Rbuild765417941dd1/lightgbm/src/build/R.def'
-- Found LibR: C:/R/RCurrent/R-4.2.2patched
-- LIBR_EXECUTABLE: C:/R/RCurrent/R-4.2.2patched/bin/x64/R.exe
-- LIBR_INCLUDE_DIRS: C:/R/RCurrent/R-4.2.2patched/include
-- LIBR_CORE_LIBRARY: C:/R/RCurrent/R-4.2.2patched/bin/x64/R.dll
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_3_0
-- Looking for CL_VERSION_3_0 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/lib/x64/OpenCL.lib (found version "3.0")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/include
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found suitable version "1.80.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Rtemp/RtmpcFC2dN/Rbuild765417941dd1/lightgbm/src/build
Successfully created build files for 'Visual Studio 17 2022'
Building lib_lightgbm
MSBuild version 17.3.1+2badb37d1 for .NET Framework
Checking Build System
Building Custom Rule D:/Rtemp/RtmpcFC2dN/Rbuild765417941dd1/lightgbm/src/CMakeLists.txt
c_api.cpp
lightgbm_R.cpp
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\vcruntime.h(61,26): warning C4083: expected ')'; found identifier '_VCRUNTIME_DISABLED_WARNINGS' (compiling source file D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\src\c_api.cpp) [D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\build\lightgbm_capi_objs.vcxproj]
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.33.31629\include\vcruntime.h(111,1): warning C4005: '_CRT_STRINGIZE': macro redefinition (compiling source file D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\src\c_api.cpp) [D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\build\lightgbm_capi_objs.vcxproj]
C:\rtools42\x86_64-w64-mingw32.static.posix\include\_mingw.h(287): message : see previous definition of '_CRT_STRINGIZE' (compiling source file D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\src\c_api.cpp) [D:\Rtemp\RtmpcFC2dN\Rbuild765417941dd1\lightgbm\src\build\lightgbm_capi_objs.vcxproj]
and a zillion errors
None of those errors, but it is still finding the wrong Boost
Nice, thanks for checking that!
Totally understood. Like I mentioned above, I didn't expect #5607 to fix the "wrong Boost" issue yet. Just trying to fix one issue at a time 😅
You may find the last line here of interest:
D:\LightGBM>Rscript build_r.R --use-gpu --boost-librarydir=D:/local/boost_1_80_0/lib64-msvc-14.3/
* checking for file 'D:/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* installing the package to build vignettes
-----------------------------------
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to D:/Rtemp/RtmpodIZGs/Rinst8a386da85e09/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 17 2022'
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.33.31629.0
-- The CXX compiler identification is MSVC 19.33.31629.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.33.31629/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- R version passed into FindLibR.cmake: 4.2.2
-- Creating R.lib and R.def
Creating 'D:/Rtemp/RtmpodIZGs/Rbuild8a383a3b7b7c/lightgbm/src/build/R.def' from 'C:/R/RCurrent/R-4.2.2patched/bin/x64/R.dll'
Successfully created 'D:/Rtemp/RtmpodIZGs/Rbuild8a383a3b7b7c/lightgbm/src/build/R.def'
-- Found LibR: C:/R/RCurrent/R-4.2.2patched
-- LIBR_EXECUTABLE: C:/R/RCurrent/R-4.2.2patched/bin/x64/R.exe
-- LIBR_INCLUDE_DIRS: C:/R/RCurrent/R-4.2.2patched/include
-- LIBR_CORE_LIBRARY: C:/R/RCurrent/R-4.2.2patched/bin/x64/R.dll
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_3_0
-- Looking for CL_VERSION_3_0 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/lib/x64/OpenCL.lib (found version "3.0")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.7/include
-- Found Boost: C:/rtools42/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.80.0/BoostConfig.cmake (found suitable version "1.80.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_LIBRARYDIR
Hi. Is this issue solved? I'm having the same problem.
PS C:\Users\drago\LightGBM> Rscript build_r.R --use-gpu --opencl-library="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.3/lib/x64/OpenCL.lib" --opencl-include-dir="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.3/include" --boost-root="C:/local/boost_1_84_0" --boost-librarydir="C:/local/boost_1_84_0/lib64-msvc-14.3" --no-build-vignettes
* checking for file 'C:/Users/drago/LightGBM/lightgbm_r/DESCRIPTION' ... OK
* preparing 'lightgbm':
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
WARNING: directory 'lightgbm/src/external_libs/compute/test' is empty
* looking to see if a 'data/datalist' file should be added
* building 'lightgbm_4.2.0.99.tar.gz'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_atomics.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/find_extrema_with_reduce.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/algorithm/detail/reduce_by_key_with_scan.hpp'
Warning in utils::tar(filepath, pkgname, compression = compression, compression_level = 9L, :
storing paths of more than 100 bytes is not portable:
'lightgbm/src/external_libs/compute/include/boost/compute/iterator/detail/get_base_iterator_buffer.hpp'
* installing to library 'C:/Users/drago/AppData/Local/R/win-library/4.3'
* installing *source* package 'lightgbm' ...
** using staged installation
** libs
installing via 'install.libs.R' to C:/Users/drago/AppData/Local/R/win-library/4.3/00LOCK-lightgbm/00new/lightgbm
Trying 'Visual Studio 17 2022'
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 6.2.9200.
-- The C compiler identification is MSVC 19.38.33133.0
-- The CXX compiler identification is MSVC 19.38.33133.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.38.33130/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- R version passed into FindLibR.cmake: 4.3.1
-- Creating R.lib and R.def
Creating 'C:/Users/drago/AppData/Local/Temp/RtmpQlSkL4/R.INSTALL1ba441c0d6956/lightgbm/src/build/R.def' from 'C:/Program Files/R/R-4.3.1/bin/x64/R.dll'
Successfully created 'C:/Users/drago/AppData/Local/Temp/RtmpQlSkL4/R.INSTALL1ba441c0d6956/lightgbm/src/build/R.def'
-- Found LibR: C:/Program Files/R/R-4.3.1
-- LIBR_EXECUTABLE: C:/Program Files/R/R-4.3.1/bin/x64/R.exe
-- LIBR_INCLUDE_DIRS: C:/Program Files/R/R-4.3.1/include
-- LIBR_CORE_LIBRARY: C:/Program Files/R/R-4.3.1/bin/x64/R.dll
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- Looking for CL_VERSION_3_0
-- Looking for CL_VERSION_3_0 - found
-- Found OpenCL: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.3/lib/x64/OpenCL.lib (found version "3.0")
-- OpenCL include directory: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v12.3/include
-- Found Boost: C:/rtools43/x86_64-w64-mingw32.static.posix/lib/cmake/Boost-1.81.0/BoostConfig.cmake (found suitable version "1.81.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Failed
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Failed
-- Looking for inet_pton
-- Looking for inet_pton - found
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_LIBRARYDIR
-- Build files have been written to: C:/Users/drago/AppData/Local/Temp/RtmpQlSkL4/R.INSTALL1ba441c0d6956/lightgbm/src/build
Successfully created build files for 'Visual Studio 17 2022'
Building lib_lightgbm
msbuild 17.8.3+195e7f5a3(.NET Framework)
Not to my knowledge.
@ruddnr has posted another reproducible example showing this issue on v4.3.0: #6289
I have not personally had time to investigate this further, and might not for a while. We're still struggling with a significant lack of maintainer availability in this project, and there are some other issues occupying my time.
I'm happy to answer questions and help with reviews, but can't devote time to testing a fix for this at the moment. We'd greatly appreciate if someone watching this thread would volunteer to try to fix this in a way that works for both Windows and Unix-likes (#5960).
As a starting point... the fix probably requires changes to some subset of these 3 files:
Description
I am trying to build LightGBM for R on Windows 10 with GPU support. I have the Boost libraries for MS VS 2022 installed under
C:\local\boost_1_78_0
as recommended, but Rtools42 comes with its own version of Boost which the Cmake finds first. When I pass--msys2
, the build works properly. However, when I do not pass it, It fails with a multitude of errors such as:I believe this is due to the build expecting VS as the compiler but finding the wrong boost libraries as per
Reproducible example
I have tried various invocations to pass the proper locations and all fail
What am I doing wrong?
Environment info
Windows 10 Home 64 Intel i7-8700K NVIDIA RTX 3080 NVIDIA CUDA 11.6 CMake 3.23 VS 2022 Boost 1.7.8 for lib64-msvc-14.3 R 4.2.0 alpha Rtools42