trivialfis
commented
2 years ago Looking at the packages, there are 2 xgboost packages, one from dmlc and another from rapids. Is is possible to remove the rapids-xgboost for now?
Closed beckernick closed 2 years ago
trivialfis
commented
2 years ago Looking at the packages, there are 2 xgboost packages, one from dmlc and another from rapids. Is is possible to remove the rapids-xgboost for now?
beckernick
commented
2 years ago Yes. When I create an environment with mamba create -n xgb-test -c conda-forge python=3.9 cudatoolkit=11.5 py-xgboost cupy, things work correctly and I get the following:
python
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:58:50)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from xgboost.compat import lazy_isinstance
>>> from xgboost.data import _is_cupy_array
>>> from xgboost._typing import DataType
>>> import cupy as cp
>>>
>>> arr = cp.array([1,2])
>>> print(_is_cupy_array(arr))
True
>>> print(lazy_isinstance(arr, "cupy", "ndarray"))
True
>>>
>>> # copied directly from github
>>> def copied_is_cupy_array(data: DataType) -> bool:
... return any(
... lazy_isinstance(data, n, "ndarray")
... for n in ("cupy.core.core", "cupy", "cupy._core.core")
... )
...
>>> print(copied_is_cupy_array(arr))The original environment is what comes by default in the current rapids nightly environment. If we need to tweak this, we can.
trivialfis
commented
2 years ago That's great news! I think the rapids-xgboost might be outdated (the recent cupy upgrade broke the _is_cupy_array function, which is fixed in the latest xgboost) and Python is confused when loading modules.
wbo4958
commented
2 years ago May I ask what's the rapids-xgboost?
trivialfis
commented
2 years ago @wbo4958 This one https://github.com/rapidsai/xgboost . It's a fork of XGBoost managed by rapids for consistent releases.
wbo4958
commented
2 years ago so, that is the source of xgboost when installing rapids from conda? like conda create -n rapids-22.08 -c rapidsai -c nvidia -c conda-forge \ rapids=22.08 python=3.9 cudatoolkit=11.5
trivialfis
commented
2 years ago @beckernick May have some insights into the packaging. My xgboost is almost always built from source ...
hcho3
commented
2 years ago @wbo4958 Yes, installing rapids from Conda will also install rapids-xgboost from the rapidsai channel.
trivialfis
commented
2 years ago Closing this now since it's caused by the environment.
As noted in https://github.com/rapidsai/cuml/issues/4903 , it appears that cupy arrays may not currently be passed the
_is_cupy_arrayinstance check, which can cause downstream errors.Environment:
Full environment: