rapidsai / cuml

cuML - RAPIDS Machine Learning Library
https://docs.rapids.ai/api/cuml/stable/
Apache License 2.0
4.25k stars 534 forks source link

Put a ceiling on cuda-python #6131

Closed bdice closed 2 weeks ago

bdice commented 2 weeks ago

This project is incompatible with newer versions of cuda-python. This puts ceilings of <=11.8.3 (CUDA 11) and <=12.6.0 (CUDA 12) on that library.

Those ceilings should be removed and replaced with != constraints once new releases of cuda-python are up that this project is compatible with.

See https://github.com/rapidsai/build-planning/issues/116 for more information.

jameslamb commented 2 weeks ago

Wheel tests here are failing like this:

E ImportError: /pyenv/versions/3.10.15/lib/python3.10/site-packages/pylibcudf/expressions.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN4cudf6detail18fixed_width_scalarIN4cuda3std3__46chrono10time_pointINS5_12system_clockENS5_8durationIlNS4_5ratioILl1ELl1000EEEEEEEE4dataEv

full stack trace (click me) ```text For more information see https://pluggy.readthedocs.io/en/stable/api_reference.html#pluggy.PluggyTeardownRaisedWarning config = pluginmanager.hook.pytest_cmdline_parse( ImportError while loading conftest '/__w/cuml/cuml/python/cuml/cuml/tests/conftest.py'. conftest.py:17: in from cuml.testing.utils import create_synthetic_dataset /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/__init__.py:17: in from cuml.internals.base import Base, UniversalBase /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/__init__.py:18: in from cuml.internals.base_helpers import BaseMetaClass, _tags_class_and_instance /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/base_helpers.py:20: in from cuml.internals.api_decorators import ( /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/api_decorators.py:24: in from cuml.internals import input_utils as iu /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/input_utils.py:20: in from cuml.internals.array import CumlArray /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/array.py:21: in from cuml.internals.global_settings import GlobalSettings /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/global_settings.py:20: in from cuml.internals.device_type import DeviceType /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/device_type.py:19: in from cuml.internals.mem_type import MemoryType /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/mem_type.py:22: in cudf = gpu_only_import("cudf") /pyenv/versions/3.10.15/lib/python3.10/site-packages/cuml/internals/safe_imports.py:362: in gpu_only_import return importlib.import_module(module) /pyenv/versions/3.10.15/lib/python3.10/site-packages/cudf/__init__.py:19: in _setup_numba() /pyenv/versions/3.10.15/lib/python3.10/site-packages/cudf/utils/_numba.py:121: in _setup_numba shim_ptx_cuda_version = _get_cuda_build_version() /pyenv/versions/3.10.15/lib/python3.10/site-packages/cudf/utils/_numba.py:16: in _get_cuda_build_version from cudf._lib import strings_udf /pyenv/versions/3.10.15/lib/python3.10/site-packages/cudf/_lib/__init__.py:4: in from . import ( avro.pyx:1: in init cudf._lib.avro ??? utils.pyx:1: in init cudf._lib.utils ??? column.pyx:1: in init cudf._lib.column ??? scalar.pyx:1: in init cudf._lib.scalar ??? /pyenv/versions/3.10.15/lib/python3.10/site-packages/pylibcudf/__init__.py:13: in from . import ( E ImportError: /pyenv/versions/3.10.15/lib/python3.10/site-packages/pylibcudf/expressions.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN4cudf6detail18fixed_width_scalarIN4cuda3std3__46chrono10time_pointINS5_12system_clockENS5_8durationIlNS4_5ratioILl1ELl1000EEEEEEEE4dataEv ```

(build link)

I suspect there was some recent change in cudf? Maybe it's notable that conda-python-tests jobs are passing and getting pylibcudf==24.12.00a297, while wheel-tests-cuml jobs are failing (running the same test code) and getting pylibcudf==24.12.00a295.

jameslamb commented 2 weeks ago

I just restarted all CI here. That cudf issue should be fixed now that there are new cudf nightly packages.

jameslamb commented 2 weeks ago

/merge

divyegala commented 2 weeks ago

/merge