search

conda-forge / pytorch_sparse-feedstock

A conda-smithy repository for pytorch_sparse.
BSD 3-Clause "New" or "Revised" License
2 stars 14 forks source link

Pytorch should be a dependency with explicit version #13

Open cmpute opened 4 years ago

cmpute commented 4 years ago

C++ based extensions of pytorch should have specific dependency on pytorch, since pytorch's ABI is not stable. So pytorch version should be pinned during the build. This also goes for other extensions like

rluria14 commented 3 years ago

@cmpute how would I go about doing this?

cmpute commented 3 years ago

Just pin the pytorch version used for these libraries would be fine

AntoinePrv commented 3 years ago

@cmpute do you know what level of pinning is required? down to the minor or bug-fix version?

With pinning, it would be good to add a variant file to build with multiple versions of pytorch.

@rluria14 I'm happy to send a PR

cmpute commented 3 years ago

@cmpute do you know what level of pinning is required? down to the minor or bug-fix version?

With pinning, it would be good to add a variant file to build with multiple versions of pytorch.

@rluria14 I'm happy to send a PR

At least minor version for my experience

hadim commented 2 years ago

I think the level of pinning should be at x.x.x (bug fix) given the below smalle tests:

# GPU
mamba create -n tmp_pyg_gpu -c conda-forge python=3.8 pytorch_geometric pytorch-gpu=1.10.2 cudatoolkit=11.1
mamba list -n tmp_pyg_gpu | grep 'pytorch\|cuda'
mamba run -n tmp_pyg_gpu python -c "import torch_geometric"  # -> WORK

# GPU (broken)
mamba create -n tmp_pyg_gpu -c conda-forge python=3.8 pytorch_geometric pytorch-gpu=1.10.0 cudatoolkit=11.1
mamba list -n tmp_pyg_gpu | grep 'pytorch\|cuda'
mamba run -n tmp_pyg_gpu python -c "import torch_geometric"  # -> FAIL

The error being:

ERROR conda.cli.main_run:execute(41): `conda run python -c import torch_geometric` failed. (See above for error)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/site-packages/torch_geometric/__init__.py", line 4, in <module>
    import torch_geometric.data
  File "/home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/site-packages/torch_geometric/data/__init__.py", line 1, in <module>
    from .data import Data
  File "/home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/site-packages/torch_geometric/data/data.py", line 9, in <module>
    from torch_sparse import SparseTensor
  File "/home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/site-packages/torch_sparse/__init__.py", line 36, in <module>
    from .storage import SparseStorage  # noqa
  File "/home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/site-packages/torch_sparse/storage.py", line 5, in <module>
    from torch_scatter import segment_csr, scatter_add
  File "/home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/site-packages/torch_scatter/__init__.py", line 13, in <module>
    torch.ops.load_library(importlib.machinery.PathFinder().find_spec(
  File "/home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/site-packages/torch/_ops.py", line 110, in load_library
    ctypes.CDLL(path)
  File "/home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/hadim/local/conda/envs/tmp_pyg_gpu/lib/python3.8/site-packages/torch_scatter/_scatter_cuda.so: undefined symbol: _ZNK2at6Tensor6deviceEv

@AntoinePrv are you planning to merge #25?

@hmaarrfk is that ok for you to pin at x.x.x given the above example? or should we stick to x.x?

hmaarrfk commented 2 years ago

Honestly, we probably had alot of breakage with older versions of pytorch 1.10.2 is probably as old as I would go.

1.11.0 i hope is in better shape with pinnings.

hadim commented 2 years ago

Honestly, we probably had alot of breakage with older versions of pytorch 1.10.2 is probably as old as I would go.

Ok this is good to know. I will stick with >=1.10.2 then.

Should we still pin those libraries pytorch_xxx?

hmaarrfk commented 2 years ago

We recently added a conda-forge wide pytorch pinning. Newely rerendered recipes that depend on pytorch will receive it.

https://github.com/conda-forge/pytorch_sparse-feedstock/blob/main/.ci_support/linux_64_c_compiler_version10cuda_compiler_version11.1cxx_compiler_version10python3.8.____cpython.yaml#L28