search

rapidsai / cloud-ml-examples

A collection of Machine Learning examples to get started with deploying RAPIDS in the Cloud
Apache License 2.0
138 stars 70 forks source link

azureml-sdk downgrades pyarrow to 3.0.0 which breaks cudf #165

Open mmccarty opened 2 years ago

mmccarty commented 2 years ago

Steps to reproduce

  1. Create a fresh RAPIDS conda environment
    conda create -n rapids-22.06 -c rapidsai -c nvidia -c conda-forge rapids=22.06 python=3.8 cudatoolkit=11.5
  2. conda activate rapids-22.06
  3. conda list | grep pyarrow shows 7.0.0 installed
  4. Launch python/ipython and import cudf should work
  5. pip install azureml-sdk
  6. Launch python/ipython and import cudf fails
  7. conda list | grep pyarrow shows 3.0.0 installed

Error:

$ python -m cudf
Traceback (most recent call last):
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/runpy.py", line 144, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/site-packages/cudf/__init__.py", line 13, in <module>
    from cudf import api, core, datasets, testing
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/site-packages/cudf/datasets.py", line 7, in <module>
    from cudf._lib.transform import bools_to_mask
  File "/home/mmccarty/miniconda3/envs/cloud-ml-examples-test/lib/python3.8/site-packages/cudf/_lib/__init__.py", line 4, in <module>
    from . import (
  File "cudf/_lib/avro.pyx", line 1, in init cudf._lib.avro
  File "cudf/_lib/column.pyx", line 1, in init cudf._lib.column
  File "cudf/_lib/scalar.pyx", line 37, in init cudf._lib.scalar
  File "cudf/_lib/interop.pyx", line 1, in init cudf._lib.interop
AttributeError: module 'pyarrow.lib' has no attribute 'MonthDayNanoIntervalArray'
mmccarty commented 2 years ago

Related to Azure/azure-sdk-for-python#25124 and Azure/azure-sdk-for-python#24644

jacobtomlinson commented 1 year ago

It looks like azureml-sdk has been superseded by azure-ai-ml so this ticket should be closed by https://github.com/rapidsai/deployment/pull/181.

cc @skirui-source