search

pytorch / functorch

functorch is JAX-like composable function transforms for PyTorch.
https://pytorch.org/functorch/
BSD 3-Clause "New" or "Revised" License
1.38k stars 102 forks source link

Torch version constraints incompatible with NVIDIA NGC PyTorch image #1072

Open timeatokai opened 1 year ago

timeatokai commented 1 year ago

Hi,

I'm using Nvidia's PyTorch NGC Docker image 22.02, which contains Torch 1.11.0a0+17540c5c. I cannot install any version of Functorch and keep the original version of Torch at the same time. Installing another version of Torch is not an option due to the danger of breaking dependencies. This problem probably appears for most or all NGC images that contain Torch older than 1.13.0, as I've only ever seen them contain custom versions of it.

The command

pip install functorch torch==1.11.0a0+17540c5 -f https://download.pytorch.org/whl/torch_stable.html

outputs (with company data removed):

Looking in indexes: ...
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Collecting functorch
  Downloading .../functorch-1.13.0-py2.py3-none-any.whl (2.1 kB)
Requirement already satisfied: torch==1.11.0a0+17540c5 in /opt/conda/lib/python3.8/site-packages (1.11.0a0+17540c5)
Requirement already satisfied: typing_extensions in /opt/conda/lib/python3.8/site-packages (from torch==1.11.0a0+17540c5) (4.0.1)
  Downloading .../functorch-0.2.1-cp38-cp38-manylinux1_x86_64.whl (20.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.7/20.7 MB 5.2 MB/s eta 0:00:00
  Downloading .../functorch-0.2.0-cp38-cp38-manylinux1_x86_64.whl (26.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 26.7/26.7 MB 4.8 MB/s eta 0:00:00
  Downloading .../functorch-0.1.1-cp38-cp38-manylinux1_x86_64.whl (21.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.5/21.5 MB 5.1 MB/s eta 0:00:00
  Downloading .../functorch-0.1.0-cp38-cp38-manylinux1_x86_64.whl (20.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.9/20.9 MB 5.1 MB/s eta 0:00:00
INFO: pip is looking at multiple versions of torch to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install functorch==0.1.0, functorch==0.1.1, functorch==0.2.0, functorch==0.2.1, functorch==1.13.0 and torch==1.11.0a0+17540c5 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested torch==1.11.0a0+17540c5
    functorch 1.13.0 depends on torch<1.13.1 and >=1.13.0
    The user requested torch==1.11.0a0+17540c5
    functorch 0.2.1 depends on torch<1.13 and >=1.12.1
    The user requested torch==1.11.0a0+17540c5
    functorch 0.2.0 depends on torch<1.13 and >=1.12
    The user requested torch==1.11.0a0+17540c5
    functorch 0.1.1 depends on torch<1.12 and >=1.11
    The user requested torch==1.11.0a0+17540c5
    functorch 0.1.0 depends on torch<1.12 and >=1.11

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

installing Functorch 0.1.0 or 0.1.1 with --no-deps doesn't work:

>>> import functorch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.8/site-packages/functorch/__init__.py", line 7, in <module>
    from . import _C
ImportError: /opt/conda/lib/python3.8/site-packages/functorch/_C.so: undefined symbol: _ZNK5torch8autograd4Node4nameEv
zou3519 commented 1 year ago

We would recommend using a torch >= 1.13.0 image if NVIDIA NGC has one. (functorch comes with the pytorch installation as of PyTorch 1.13.0).

If you really need to get functorch working with an older version of PyTorch: the procedure is to find the commit hash of PyTorch that the PyTorch build uses, find the nearest functorch commit hash (in terms of date/time), and then:

pip install git+https://github.com/pytorch/functorch.git@commit_hash