search

torch-points3d / torch-points-kernels

Pytorch kernels for spatial operations on point clouds
MIT License
96 stars 25 forks source link

Create Conda Package #89

Closed CCInc closed 2 years ago

Stakhan commented 2 years ago

I'll take care of this one! :)

CCInc commented 2 years ago

@Stakhan Based on your work I updated our existing code.

You can see the runs here: https://github.com/CCInc/torch-points-kernels/actions/runs/2592145686 PR: https://github.com/torch-points3d/torch-points-kernels/pull/96 Package: https://anaconda.org/csl170000/torch-points-kernels-test2

If you have a chance, you can test the packages with:

conda install -c csl170000 torch-points-kernels-test2

CCInc commented 2 years ago

I haven't seen any instances of cuda packages without the cuda code being built, but if you find any, let me know.

Stakhan commented 2 years ago

Well done @CCInc. I'm curious, what were the missing parts for it to properly compile during conda build on Linux? I recall it wouldn't detect the presence of CUDA.

Stakhan commented 2 years ago

Here are a few combinations I tried to install with conda on my side:

Error for the combinations with Python 3.10:

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package cudatoolkit conflicts for:
torch-points-kernels-test2 -> cudatoolkit[version='10.2.*|11.3.*|11.6.*']
cudatoolkit=11.3
torch-points-kernels-test2 -> pytorch=1.12 -> cudatoolkit[version='>=10.2,<10.3|>=11.3,<11.4|>=11.6,<11.7|>=11.5,<11.6|>=11.1,<11.2']

Package python conflicts for:
torch-points-kernels-test2 -> python[version='>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0']
torch-points-kernels-test2 -> numba -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.6,<3.7.0a0|>=3.5,<3.6.0a0']
python=3.10

Package _libgcc_mutex conflicts for:
python=3.10 -> libgcc-ng[version='>=7.5.0'] -> _libgcc_mutex[version='*|0.1',build=main]
cudatoolkit=11.3 -> libgcc-ng[version='>=9.3.0'] -> _libgcc_mutex[version='*|0.1',build=main]

Package pytorch conflicts for:
pytorch=1.12.0
torch-points-kernels-test2 -> pytorch[version='1.10.*|1.11.*|1.12.*']

Package libgcc-ng conflicts for:
torch-points-kernels-test2 -> numba -> libgcc-ng[version='>=11.2.0|>=7.2.0|>=7.3.0|>=7.5.0|>=9.3.0']
pytorch=1.12.0 -> cudatoolkit[version='>=11.3,<11.4'] -> libgcc-ng[version='>=7.2.0|>=7.3.0|>=7.5.0|>=9.3.0']
cudatoolkit=11.3 -> libgcc-ng[version='>=9.3.0']
python=3.10 -> libgcc-ng[version='>=7.5.0']
python=3.10 -> bzip2[version='>=1.0.8,<2.0a0'] -> libgcc-ng[version='>=11.2.0|>=7.2.0|>=7.3.0']

Package setuptools conflicts for:
torch-points-kernels-test2 -> numba -> setuptools
python=3.10 -> pip -> setuptools

Package libuuid conflicts for:
python=3.10 -> libuuid[version='>=1.0.3,<2.0a0']
pytorch=1.12.0 -> python[version='>=3.10,<3.11.0a0'] -> libuuid[version='>=1.0.3,<2.0a0']

Package _openmp_mutex conflicts for:
cudatoolkit=11.3 -> libgcc-ng[version='>=9.3.0'] -> _openmp_mutex[version='>=4.5']
python=3.10 -> libgcc-ng[version='>=7.5.0'] -> _openmp_mutex[version='>=4.5']
torch-points-kernels-test2 -> numba -> _openmp_mutex[version='>=4.5']The following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.31=0
  - feature:|@/linux-64::__glibc==2.31=0
  - cudatoolkit=11.3 -> libgcc-ng[version='>=9.3.0'] -> __glibc[version='>=2.17']
  - python=3.10 -> libgcc-ng[version='>=7.5.0'] -> __glibc[version='>=2.17']

Your installed version is: 2.31
CCInc commented 2 years ago

I have no idea what the issue was. All I did was copied the cuda scripts from pytorch_sparse and updated the channels it used to build (maybe that made a difference?)

conda build . -c pytorch -c default -c nvidia --output-folder "$HOME/conda-bld"

The only thing I didn't do was get Windows to compile - not sure if that's something you want to tackle but it's pretty low priority for me 😉 I have not had good luck with Windows conda packages.

CCInc commented 2 years ago

Thanks for testing! It looks like the python 3.10 combination had some issues for pytorch_sparse, so I left it disabled:

          # There is a weird `glibc=2.27` bug going on for this combination:
          - os: ubuntu-18.04
            python-version: '3.10'

If it's requested, we can try to debug it and see what the issue is, I think we're okay supporting just 3.7-3.9 for now, but let me know if you have any thoughts.

CCInc commented 2 years ago

It would be nice to have packages for older versions of pytorch (1.8 and 1.9) but I think we should try to focus updating tp3d and its dependencies to the latest versions of pytorch, especially because we've seen some big performance improvements in certain areas with the latest versions of pytorch.

CCInc commented 2 years ago

This is now released on the official torch-points3d channel. You can install with:

conda install -c torch-points3d torch-points-kernels

Let me know how it works for everyone.

Stakhan commented 2 years ago

Hi @CCInc, Sorry for the late reply, was in holidays for the last weeks. I also think it's okay to only support python 3.8-3.9 for the moment, and have only ubuntu covered.

Works fine on my side (ubuntu 20.04, Python 3.8, CUDA 11.3, PyTorch 1.12)

Great job! 😎