search

microsoft / LightGBM

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
https://lightgbm.readthedocs.io/en/latest/
MIT License
16.72k stars 3.84k forks source link

LightGBMError: GPU Tree Learner was not enabled in this build. Even when LightGBM was compiled successfully with CMake option -DUSE_GPU=1 #5928

Closed studiocardo closed 1 year ago

studiocardo commented 1 year ago

Description

In Google Colab, I did:

! git clone --recursive https://github.com/Microsoft/LightGBM
! cd LightGBM && rm -rf build && mkdir build && cd build && cmake -DUSE_GPU=1 .. && make -j4 && cd ..

Compilation was completed successfully to 100%.

When I ran 

params = {"objective" : 'regression',              # "rmse" is an alias to regression, L2 loss
          "boosting" : "gbdt",
          "learning_rate" : 0.02,
          "num_iterations" : 500,            # default=100
          "max_depth" : 12,
          "num_leaves" : 256,                # recommended num_leaves = 2**max_depth
          "min_data_in_leaf" : 5,           # default = 20
          "device" : 'gpu'
         }

model = lgb.train(params,
                   lgbm_train,
                   verbose_eval=10,
                   keep_training_booster=True)

I received the following error message:

LightGBMError: GPU Tree Learner was not enabled in this build. Please recompile with CMake option -DUSE_GPU=1

What happened? Didn't I just compile LightGBM with -DUSE_GPU=1 successfully? Why doesn't it pick up the GPU option?

Reproducible example

! git clone --recursive https://github.com/Microsoft/LightGBM
! cd LightGBM && rm -rf build && mkdir build && cd build && cmake -DUSE_GPU=1 .. && make -j4 && cd ..
params = {"objective" : 'regression',              # "rmse" is an alias to regression, L2 loss
          "device" : 'gpu'         }

model = lgb.train(params,
                   lgbm_train,
                   verbose_eval=10,
                   keep_training_booster=True)

Environment info

Google Colab:

LightGBM version or commit hash: 3.3.5

Command(s) you used to install LightGBM

! git clone --recursive https://github.com/Microsoft/LightGBM
! cd LightGBM && rm -rf build && mkdir build && cd build && cmake -DUSE_GPU=1 .. && make -j4 && cd ..
! cd LightGBM && rm -rf build && mkdir build && cd build && cmake -DUSE_GPU=1 .. && make -j4 && cd ..
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")  
-- Looking for CL_VERSION_3_0
-- Looking for CL_VERSION_3_0 - not found
-- Looking for CL_VERSION_2_2
-- Looking for CL_VERSION_2_2 - found
-- Found OpenCL: /usr/lib/x86_64-linux-gnu/libOpenCL.so (found version "2.2")
-- OpenCL include directory: /usr/include
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found suitable version "1.71.0", minimum required is "1.56.0") found components: filesystem system
-- Performing Test MM_PREFETCH
-- Performing Test MM_PREFETCH - Success
-- Using _mm_prefetch
-- Performing Test MM_MALLOC
-- Performing Test MM_MALLOC - Success
-- Using _mm_malloc
-- Configuring done
-- Generating done
-- Build files have been written to: /content/LightGBM/build
[ 1%] Building CXX object CMakeFiles/lightgbm_capi_objs.dir/src/c_api.cpp.o 
[ 5%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/boosting.cpp.o 
[ 5%] Building CXX object CMakeFiles /lightgbm_objs.dir/src/boosting/cuda/cuda_score_updater.cpp.o 
[ 7%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt.cpp.o 
[ 9%] Building CXX object CMakeFiles/lightgbm_objs.dir /src/boosting/gbdt_model_text.cpp.o 
[ 10%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/gbdt_prediction.cpp.o 
[ 12%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/prediction_early_s top .cpp.o 
[ 14%]Building CXX object CMakeFiles/lightgbm_objs.dir/src/boosting/sample_strategy.cpp.o 
[ 16%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/cuda/cuda_utils.cpp.o 
[ 18%] Building CXX object CMakeFiles/lightgbm_objs .dir/src/io/bin.cpp.o 
[ 20%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/config.cpp.o 
[ 21%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io /config_auto.cpp.o
[ 21%] Built target lightgbm_capi_objs
[ 23%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/cuda/cuda_column_data.cpp.o 
[ 25%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/cuda/cuda_metadata.cpp.o 
[ 27 %] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/cuda/cuda_row_data.cpp.o 
[ 29%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/cuda/cuda_tree.cpp.o 
[ 30%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/dataset.cpp.o 
[ 32%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/dataset_loader.cpp.o 
[ 34%] Building CXX object CMakeFiles/lightgbm_objs .dir/src/io/file_io.cpp.o 
[ 36%]Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/json11.cpp.o 
[ 38%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/metadata.cpp.o 
[ 40%] Building CXX object CMakeFiles/lightgbm_objs .dir/src/io/parser.cpp.o 
[ 41%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io/train_share_states.cpp.o 
[ 43%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/io /tree.cpp.o 
[ 45%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/metric/cuda/cuda_binary_metric.cpp.o 
[ 47%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/metric/cuda/cuda_pointwise_metric .cpp.o 
[ 49%]Building CXX object CMakeFiles/lightgbm_objs.dir/src/metric/cuda/cuda_regression_metric.cpp.o 
[ 50%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/metric/dcg_calculator.cpp.o 
[ 52%] Building CXX object CMakeFiles /lightgbm_objs.dir/src/metric/metric.cpp.o 
[ 54%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/network/linker_topo.cpp.o 
[ 56%] Building CXX object CMakeFiles/lightgbm_objs.dir/src /network/linkers_mpi.cpp.o 
[ 58%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/network/linkers_socket.cpp.o 
[ 60%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/network/network.cpp .o 
[ 61%]Building CXX object CMakeFiles/lightgbm_objs.dir/src/objective/cuda/cuda_binary_objective.cpp.o 
[ 63%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/objective/cuda/cuda_multiclass_objective.cpp.o 
[ 65%] Building C XX object CMakeFiles/lightgbm_objs.dir/src/objective/cuda/cuda_rank_objective.cpp.o 
[ 67%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/objective/cuda/cuda_regression_objective.cpp.o 
[ 69%] Building CXX object CMake files /lightgbm_objs.dir/src/objective/objective_function.cpp.o 
[ 70%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/cuda/cuda_best_split_finder.cpp.o 
[ 72%]Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/cuda/cuda_data_partition.cpp.o 
[ 74%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/cuda/cuda_histogram_constructor.cpp.o 
[ 76%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/cuda/cuda_leaf_splits.cpp.o 
[ 78%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/cuda/cuda_single_gpu_tree_learner.cpp.o 
[ 80%] Building CXX object CMakeFiles /lightgbm_objs.dir/src/treelearner/data_parallel_tree_learner.cpp.o 
[ 81%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/feature_parallel_tree_learner.cpp.o 
[ 83%]Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/gpu_tree_learner.cpp.o 
In file included from /usr/include/CL/cl.h:32 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/cl.hpp:19 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/config.hpp:16 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/buffer.hpp:14 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/core.hpp:18 ,
                 from /content/LightGBM/src/treelearner/gpu_tree_learner.h:33 ,
                 from /content/LightGBM/src/treelearner/parallel_tree_learner.h:15 ,
                 from /content/LightGBM/src/treelearner/data_parallel_tree_learner.cpp:9 :
 /usr/include/CL/cl_version.h:34:104:  note: #pragma message: cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)
   34 | #pragma message("cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)" ) 
      |                                                                                                        ^ 
In file included from /usr/include/CL/cl.h:32 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/cl.hpp:19 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/config.hpp:16 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/buffer.hpp:14 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/core.hpp:18 ,
                 from /content/LightGBM/src/treelearner/gpu_tree_learner.h:33 ,
                 from /content/LightGBM/src/treelearner/parallel_tree_learner.h:15 ,
                 from /content/LightGBM/src/treelearner/feature_parallel_tree_learner.cpp:8 :
 /usr/include/CL/cl_version.h:34:104:  note: #pragma message: cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)
   34 | #pragma message("cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)" ) 
      |                                                                                                        ^ 
In file included from /usr/include/CL/cl.h:32 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/cl.hpp:19 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/config.hpp:16 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/buffer.hpp:14 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/core.hpp:18 ,
                 from /content/LightGBM/src/treelearner/gpu_tree_learner.h:33 ,
                 from /content/LightGBM/src/treelearner/gpu_tree_learner.cpp:7 :
 /usr/include/CL/cl_version.h:34:104:  note: #pragma message: cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)
   34 | #pragma message("cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)" ) 
      |                                                                                                        ^ 
[ 85%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/gradient_discretizer.cpp.o 
[ 87%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/linear_tree_learner.cpp.o 
[ 89%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/serial_tree_learner.cpp.o 
[ 90%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/tree_learner.cpp.o 
In file included from /usr/include/CL/ cl.h:32 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/cl.hpp:19 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/config.hpp:16 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/buffer.hpp:14 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/core.hpp:18 ,
                 from /content/LightGBM/src/treelearner/gpu_tree_learner.h:33 ,
                 from /content/LightGBM/src/treelearner/tree_learner.cpp:7 :
 /usr/include/CL/cl_version.h:34:104:  note: #pragma message: cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)
   34 | #pragma message("cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)" ) 
      |                                                                                                        ^ 
[ 92%] Building CXX object CMakeFiles/lightgbm_objs.dir/src/treelearner/voting_parallel_tree_learner.cpp.o 
In file included from /usr/include/CL/cl.h:32 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/cl.hpp:19 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/config.hpp:16 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/buffer.hpp:14 ,
                 from /content/LightGBM/external_libs/compute/include/boost/compute/core.hpp:18 ,
                 from /content/LightGBM/src/treelearner/gpu_tree_learner.h:33 ,
                 from /content/LightGBM/src/treelearner/parallel_tree_learner.h:15 ,
                 from /content/LightGBM/src/treelearner/voting_parallel_tree_learner.cpp:11 :
 /usr/include/CL/cl_version.h:34:104:  note: #pragma message: cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)
   34 | #pragma message("cl_version.h: CL_TARGET_OPENCL_VERSION is not defined. Defaulting to 220 (OpenCL 2.2)" ) 
      |                                                                                                        ^
[ 92%] Built target lightgbm_objs
[ 94%] Building CXX object CMakeFiles/lightgbm.dir/src/main.cpp.o 
[ 96%] Building CXX object CMakeFiles/lightgbm.dir/src/application/application.cpp.o 
[ 98%] Linking CXX shared library /content/LightGBM/lib_lightgbm.so
[ 98%] Built target _lightgbm
[100%] Linking CXX executable /content/LightGBM/lightgbm
[100%] Built target lightgbm
jameslamb commented 1 year ago

Thanks for using LightGBM.

First, please note that I've reformatted the text in your question to make it clearer. If you're new to GitHub, please read https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax to learn how to format text in GitHub-flavored markdown.

What happened? Didn't I just compile LightGBM with -DUSE_GPU=1 successfully? Why doesn't it pick up the GPU option?

It appears you're trying to use the LightGBM Python package, lightgbm.

The commands you've provided only compiled the C++ shared library, lib_lightgbm.so. To use it in the Python package, you also need to build and install the Python package.

Either like this:

git clone --recursive https://github.com/Microsoft/LightGBM
cd LightGBM
rm -rf ./build
mkdir build
cd ./build
cmake -DUSE_GPU=1 ..
make -j4
cd ..
sh ./build-python.sh install --precompile

Or like this (to do compilation + installation of the Python package in one step):

sh ./build-python.sh install --gpu

This is documented at https://github.com/microsoft/LightGBM/tree/master/python-package#install-from-github.

studiocardo commented 1 year ago

Hi James

Thanks for your help. Google seems to prefer directing all my queries to https://lightgbm.readthedocs.io/en/stable/Installation-Guide.html#id17...

And the information seems a little contradictory at times. But if the GPU version is available in binary for Windows without having to recompile it first, I'll give that a try first...

Thx, SH

jameslamb commented 1 year ago

And the information seems a little contradictory at times

We'd welcome pull requests fixing whatever information you find to be "contradictory". Or, if you can be specific about what is confusing, misleading, or inaccurate, I'd be happy to make the relevant changes in the docs.

it if the GPU version is available in binary for Windows without having to recompile it first

It is. That support was added a few years ago, in #3144. I didn't mention that option earlier because in your initial post, you mentioned cloning LightGBM from GitHub... so I assumed you wanted to build from the latest development version.

If using the most recent release of lightgbm is acceptable for you, then yes you can just pip install lightgbm on Windows and pass "device": "gpu" to use the GPU-based implementation of LightGBM.

I'm also surprised to learn that a Windows-only solution would work for you, since your report above contains log lines like the following with Unix-style paths:

-- Found OpenCL: /usr/lib/x86_64-linux-gnu/libOpenCL.so (found version "2.2")

And since as far as I know, Google Colab only supports Linux.

studiocardo commented 1 year ago

Thx James, if GPU support readily available in binary form, I will try to install it again on my Win11 laptop (it's a pain finding stable Linux/laptop combination...). I did attempt to compile LightGBM w/ VisualStudio with instruction from https://lightgbm.readthedocs.io/en/stable/Installation-Guide.html#id17 a month ago, but I wasn't able to get it to work... So I turned to Colab/Sage/Lambda, and none of them is a straight forward python installation... I will try solution from #3144, and I hope it works. I will update you on the progress. Thx again!

jameslamb commented 1 year ago

I did attempt to compile LightGBM w/ VisualStudio with instruction from https://lightgbm.readthedocs.io/en/stable/Installation-Guide.html#id17 a month ago, but I wasn't able to get it to work

If you can provide a reproducible example, including all commands you ran and the errors you encountered, we'd be happy to help. Every commit to master here is tested across 20+ combinations of compiler, operating system, and Python version, including 3 different versions of Visual Studio.

The next release (#5153) will also include integrated-OpenCL wheels for Linux, where you can just pip install lightgbm and then pass "device": "gpu".

wanggxaaa commented 1 year ago

I have also encountered this issue. I am a novice and need to use the LGBM package when testing my. ipynb code. However, the Python environment I am using can only be imported correctly through pip installation. After pip installation, the prompt 'GPU Tree Learner was not enabled in this build.' Please reconcile with CMake option - DUSE_ GPU=1 ', I don't know how to recompile packages downloaded through pip, and I am not able to reference packages downloaded from github into my environment,I can't import lightgbm through this download method. Can you help me?

jameslamb commented 1 year ago

I am not able to reference packages downloaded from github into my environment,I can't import lightgbm through this download method

What does "not able to reference packages downloaded from GitHub in my environment" mean specifically?

If, as I suspect, you just mean that you only have access to an IPython notebook and not a shell, you should actually be able to run arbitrary shell commands using ! or the %%script magic (documented here, like this:

%%script
git clone --recursive https://github.com/microsoft/LightGBM.git
sh ./build-python.sh install --gpu

Or if you mean that you don't have git available and don't have permissions to install it, you can install LightGBM's source code from PyPI and build a GPU-enabled version of it.

!pip install --no-binary lightgbm 'lightgbm==3.3.5' --install-option=--gpu

Note that I intentionally pinned to v3.3.5 in that example, as --install-option support was removed in recent versions of pip and therefore won't work with v4.0.0 of lightgbm when that's released (#5153).

Also note that the pip install approach assumes that you have CMake and a C++ compiler available in the environment you're working in.

If neither of these work for you, please describe precisely that you mean by "not able to reference packages downloaded from Github".

github-actions[bot] commented 1 year ago

This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!

github-actions[bot] commented 1 year ago

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.