Installing torchvision for ppc64le IBM architecture

brando90 commented 4 years ago

I am in a weird scenario were I am forced to use torch 1.3.1 (due to hardware see: https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/). I read from the pytorch docs that it's corresponding version of torchvision is 0.4.1 (https://pypi.org/project/torchvision/):

Installation

We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.
Installation
We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.

torch torchvision python
master / nightly  master / nightly    >=3.6
1.5.0 0.6.0   >=3.5
1.4.0 0.5.0   ==2.7, >=3.5, <=3.8
1.3.1 0.4.2   ==2.7, >=3.5, <=3.7
1.3.0 0.4.1   ==2.7, >=3.5, <=3.7
1.2.0 0.4.0   ==2.7, >=3.5, <=3.7
1.1.0 0.3.0   ==2.7, >=3.5, <=3.7
<=1.0.1   0.2.2   ==2.7, >=3.5, <=3.7

but for some reason I have the wrong version of it:

torchvision 0.2.2 pypi_0 pypi

is there a way to install the right version of torchvision?

What I've tried:

First I tried force installing the right version with conda. Conda couldn't find the version of torchvision that I need:

$ conda install torchvision==0.4.2
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - torchvision==0.4.2

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Then I proceeded to try to install it regardless with pip

$ pip install torchvision==0.4.2
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement torchvision==0.4.2 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3)
ERROR: No matching distribution found for torchvision==0.4.2

got an error too.

Is there anything else to try?

I tried but it failed:

conda install torchvision==0.4.2 -c pytorch

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - torchvision==0.4.2

Current channels:

  - https://conda.anaconda.org/pytorch/linux-ppc64le
  - https://conda.anaconda.org/pytorch/noarch
  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

crossposted SO: https://discuss.pytorch.org/t/force-installing-torchvision/97279
crossposted reddit pytorch: https://www.reddit.com/r/pytorch/comments/iyf2qn/force_installing_torchvision/
crossposted reddit ibm: https://www.reddit.com/r/IBM/comments/iyhzex/force_installing_torchvision_042_when_i_am_forced/
real problem is installing torchmeta: https://github.com/tristandeleu/pytorch-meta/issues/95
https://www.ibm.com/mysupport/s/forumsquestion?id=0D50z00006gaxV9CAI
quora: https://www.quora.com/unanswered/How-does-one-install-specific-Python-packages-in-Conda-from-IBM-architectures
reddit ibm2: https://www.reddit.com/r/newIBM/comments/iyij10/force_installing_torchvision_042_when_i_am_forced/
gitissue for ibm: https://github.com/IBM/powerai/issues/268

cc @fmassa @vfdev-5

brando90 commented 4 years ago

https://github.com/IBM/powerai/issues/268

zou3519 commented 4 years ago

The torchvision issue tracker might be a better place for this: https://github.com/pytorch/vision/issues

vfdev-5 commented 4 years ago

@brando90 interesting issue. I just tried in a clean conda env (x86 linux) to install both pytorch and torchvision :

conda install pytorch=1.3.1 torchvision -c pytorch

and it proposes to install the correct version:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cffi-1.14.3                |   py36he30daa8_0         222 KB
    intel-openmp-2020.2        |              254         786 KB
    mkl-2020.2                 |              256       138.3 MB
    mkl_fft-1.2.0              |   py36h23d657b_0         149 KB
    ninja-1.10.1               |   py36hfd86e86_0         1.4 MB
    numpy-1.19.1               |   py36hbc911f0_0          21 KB
    numpy-base-1.19.1          |   py36hfa32c7d_0         4.1 MB
    olefile-0.46               |           py36_0          48 KB
    pillow-7.2.0               |   py36hb39fc2d_0         619 KB
    pycparser-2.20             |             py_2          94 KB
    pytorch-1.3.1              |py3.6_cuda10.1.243_cudnn7.6.3_0       428.0 MB  pytorch
    torchvision-0.4.2          |       py36_cu101        14.8 MB  pytorch

Can you try to run the command with pytorch=1.3.1 which normally shouldn't reinstall existing pytorch ?

EDIT: Probably, the issue is with linux-ppc64le architecture and the only noarch binary is 0.2.2 according to anaconda: https://anaconda.org/pytorch/torchvision

Can you try to build it from sources, from github branch v0.4.2 : https://github.com/pytorch/vision/tree/v0.4.2 ?

pmeier commented 4 years ago

@vfdev-5 Your hunch is correct. As of torchvision==0.3 the provided binaries are precompiled for a few selected platforms. Thus, if you try to install with an unsupported platform it falls back to the last version without platform requirements, i. e. torchvision==0.2.2.

owlesia commented 1 year ago

In order to install the correct version of torchvision on PPC64le I found it easy to just install the IBM MLDL framework which includes correct versions of pytorch and torchvision-base:

conda install powerai

escanore981 commented 1 year ago

PackagesNotFoundError: The following packages are not available from current channels: powerai

pytorch / vision

Installing torchvision for ppc64le IBM architecture #2709

Installation