heavyai / heavyai

Python Data Science package for HeavyDB.
https://heavyai.readthedocs.io/en/latest/
Apache License 2.0
20 stars 19 forks source link

install pymapd and cudf for GPU Dataframe support (conda-only) not working? #27

Closed kimgiftww closed 2 years ago

kimgiftww commented 2 years ago

I tried the below command after a fresh miniconda installation (https://repo.anaconda.com/miniconda/Miniconda3-py37_4.11.0-Linux-x86_64.sh)

conda create -n omnisci-gpu -c rapidsai -c nvidia -c conda-forge \ -c defaults cudf=0.18 python=3.7 cudatoolkit=11.0 pyomnisci

as mentioned in here

https://github.com/heavyai/pyomnisci/blob/master/README.rst

and not successful with the following conflict:

Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: / Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abort. failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package numba conflicts for: cudf -> numba[version='>=0.40|>=0.41|>=0.45.1|>=0.46.0|>=0.48.0|>=0.49.0|>=0.53.1|>=0.54'] cudf -> rmm[version='>=22.2.0,<22.3.0a0'] -> numba[version='>=0.49']

Package libgcc-ng conflicts for: python=3.8 -> libgcc-ng[version='>=7.3.0|>=7.5.0|>=9.3.0|>=9.4.0'] python=3.8 -> openssl[version='>=3.0.0,<4.0a0'] -> libgcc-ng[version='>=10.3.0|>=4.9|>=7.2.0']

Package _openmp_mutex conflicts for: python=3.8 -> libgcc-ng[version='>=9.4.0'] -> _openmp_mutex[version='>=4.5'] cudatoolkit -> libgcc-ng[version='>=9.4.0'] -> _openmp_mutex[version='>=4.5'] cudf -> numba[version='>=0.54'] -> _openmp_mutex[version='>=4.5'] pyomnisci -> numba[version='>=0.49'] -> _openmp_mutex[version='>=4.5']

Package libstdcxx-ng conflicts for: python=3.8 -> libstdcxx-ng[version='>=7.3.0|>=7.5.0|>=9.3.0|>=9.4.0'] python=3.8 -> libffi[version='>=3.2.1,<3.3.0a0'] -> libstdcxx-ng[version='>=4.9|>=7.2.0']

Package cudatoolkit conflicts for: cudatoolkit cudf -> cupy[version='>=9.5.0,<11.0.0a0'] -> cudatoolkit[version='10.0|10.0.|10.1|10.1.|10.2|10.2.|11.0|11.0.|11.1|11.1.|>=11.2,<12|9.2|9.2.|>=11.0,<=11.6|>=11.0,<=11.5|>=11.0,<11.2|>=11.2,<12.0a0|11.4|11.4.*|>=10.0.130,<10.1.0a0|>=9.2,<9.3.0a0|>=10.1.168,<10.2.0a0'] cudf -> cudatoolkit[version='>=10.1.243,<10.2.0a0|>=11,<12.0a0|>=11.2.72,<11.3.0a0|>=11.0.221,<11.1.0a0|>=11.4.1,<11.5.0a0|>=11.4.0,<11.5.0a0|>=10.2.89,<10.3.0a0']

Package python conflicts for: pyomnisci -> python[version='>=3.7|>=3.7,<3.9'] cudf -> python[version='>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0'] pyomnisci -> numba[version='>=0.49'] -> python[version='2.7.|3.5.|3.6.|>=2.7,<2.8.0a0|>=2.7|>=3.10,<3.11.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.6,<3.7.0a0|>=3.6|>=3.5,<3.6.0a0|3.4.|>=3.7.1,<3.8.0a0'] python=3.8 cudf -> cachetools -> python[version='2.7.|3.4.|3.5.|3.6.|>=3.5|>=3.5,<3.6|>=3.10,<3.11.0a0|>=2.7,<2.8.0a0|>=3.6|>=3.7|>=3.5,<3.6.0a0|>=2.7|>=3.7.1,<3.8.0a0|3.8.|3.9.|3.7.*']

Package numpy conflicts for: cudf -> numpy cudf -> cupy[version='>=9.5.0,<11.0.0a0'] -> numpy[version='1.15.*|>=1.11.3,<2.0a0|>=1.14.6,<1.18.0|>=1.14.6,<2.0a0|>=1.15.4,<2.0a0|>=1.16.5,<2.0a0|>=1.16.6,<2.0a0|>=1.17|>=1.18|>=1.21.5,<2.0a0|>=1.18.5,<2.0a0|>=1.19.5,<2.0a0|>=1.21.2,<2.0a0|>=1.21.4,<2.0a0|>=1.17.5,<2.0a0|>=1.19.4,<2.0a0|>=1.19.2,<2.0a0|>=1.18.4,<2.0a0|>=1.18.1,<2.0a0|>=1.20.3,<2.0a0|>=1.20.2,<2.0a0|>=1.16,<2.0a0|>=1.9.0|>=1.16|>=1.15|>=1.16,<1.20.0a0|>=1.13.3,<2.0a0|>=1.12.1,<2.0a0|>=1.14,<1.20.0a0|>=1.9.3,<2.0a0']

Package _libgcc_mutex conflicts for: cudatoolkit -> libgcc-ng[version='>=9.4.0'] -> _libgcc_mutex[version='|0.1',build='main|main|conda_forge'] python=3.8 -> libgcc-ng[version='>=9.4.0'] -> _libgcc_mutex[version='|0.1',build='main|main|conda_forge']

Package pyarrow conflicts for: cudf -> libcudf=0.5.1 -> pyarrow=0.12.0 cudf -> pyarrow[version='0.12.1.|0.14.1.|0.15.0.|0.17.1.|>=1.0.1,<1.0.2.0a0|>=4.0.1,<4.0.2.0a0|>=5.0.0,<5.0.1.0a0',build=*cuda]

Package pandas conflicts for: cudf -> pandas[version='>=0.23.4|>=0.24.2,<0.25|>=0.25,<0.26|>=1.0,<1.2.0dev0|>=1.0,<1.3.0dev0|>=1.0,<1.4.0dev0|>=1.0,<=1.2.4'] cudf -> pyarrow[version='>=1.0.1,<1.0.2.0a0'] -> pandasThe following specifications were found to be incompatible with your system:

Your installed version is: 2.31

Could you please update the recommended package version in the readme as it not seem to be updated and perhaps not compatible now? I'm using Ubuntu 20.04

kimgiftww commented 2 years ago

ContextualVersionConflict: (pyarrow 5.0.0 (/opt/conda/envs/rapids/lib/python3.8/site-packages), Requirement.parse('pyarrow==3.0.0'), {'pyomnisci'})

Is this one of the reasons I'm facing this error as the version of pyarrow (required by rapids) is not compatible with pyomnisci?

tupui commented 2 years ago

Hi @kimgiftww, we released a new version on PyPi and conda will follow. There should be no problems now.

kimgiftww commented 2 years ago

https://pypi.org/project/heavyai

I found it thanks.

Tested that it is even compatible with the latest docker from rapids.ai

docker pull rapidsai/rapidsai-core-dev-nightly:22.04-cuda11.5-devel-ubuntu20.04-py3.8
docker run --gpus all --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 \
    rapidsai/rapidsai-core-dev-nightly:22.04-cuda11.5-devel-ubuntu20.04-py3.8