dependencies incompatibilities between h2o4gpu and cudf

[retimadangol]

Description

• There're dependencies incompatibilities between h2o4gpu and cudf
• need to update it and also support multiple versions

when trying to get a conda environment with Nvidia’s cudf package and H2o’s h2o4gpu package it fails

Error

$ conda env create -f ./h2o4gpuwcudf.yml Solving environment: failed

UnsatisfiableError: The following specifications were found to be in conflict:

• cudf=0.10 -> cython[version='>=0.29,<0.30'] -> python_abi=[build=*_cp27mu] -> pypy[version='<0a0']
• cudf=0.10 -> libcudf[version='>=0.10.0,<0.11.0a0'] -> arrow-cpp[version='0.14.1.*,>=0.14.1,<0.15.0a0'] -> boost-cpp[version='>=1.70.0,<1.70.1.0a0'] -> libboost[version='<0']
• cudf=0.10 -> numba[version='>=0.45.1'] -> llvmlite=0.29
• cudf=0.10 -> pyarrow=0.14.1
• h2o4gpu-cuda10 -> awscli==1.15.6 -> python_abi=[build=*_cp27mu] -> pypy[version='<0a0']
• h2o4gpu-cuda10 -> cython==0.29.3
• h2o4gpu-cuda10 -> feather-format==0.4.0 -> pyarrow
• h2o4gpu-cuda10 -> llvmlite==0.23.0
• h2o4gpu-cuda10 -> numba==0.38.0
• h2o4gpu-cuda10 -> scikit-learn==0.20.2 Use "conda info " to see the dependencies for each package.

environment file, h204gpuwcudf.yml.

name: h2o4gpuwcudf channels:

• h2oai
• rapidsai
• nvidia
• numba
• conda-forge
• defaults dependencies:
• cudf=0.10
• cudatoolkit=10.0
• h2o4gpu-cuda10
• jupyter
• matplotlib
• python=3.6
• tornado=4.5.3
• pip:
  • jupyter_client

    - https://s3.amazonaws.com/h2o-release/h2o4gpu/releases/stable/ai/h2o/h2o4gpu/0.3-cuda10/h2o4gpu-0.3.2-cp36-cp36m-linux_x86_64.whl

However, uncommenting out the wheel file appears to work with these minimal dependencies

https://support.h2o.ai/a/tickets/96469

[sh1ng]

I don't know why but the file contains both h2o4gpu-cuda conda package and pip package from s3. If I remove h2o4gpu-cuda10 package and replace pip with bleeding edge release https://s3.amazonaws.com/h2o-release/h2o4gpu/releases/bleeding-edge/ai/h2o/h2o4gpu/0.3-cuda10/h2o4gpu-0.3.2-cp36-cp36m-linux_x86_64.whl it works.

I'm going to release version 0.4 this week.

[pinskeep]

The file has both because I was trying it both ways, commenting one or the other out. In the version above the s3 one was commented out to show the error. It was mentioned in the issue that using the wheel seemed to work. Thanks for your help with this as it helps with other cuda libraries as well.

[sh1ng]

I need to make some changes about multi-gpu support of xgboost. The release is postponed.

[pinskeep]

Thanks, looking forward to it.

[NidhiMehta]

@sh1ng any update on this . When will we have the new release out. thanks.

[sh1ng]

@NidhiMehta it was released yesterday.

Thank you for your patience.

[pinskeep]

The updated conda environment seems to be working for us. Thank you!