search

IBM / pytorch-large-model-support

Large Model Support in PyTorch
Apache License 2.0
131 stars 19 forks source link

How does one install pytorch on ppc64le? #6

Closed brando90 closed 3 years ago

brando90 commented 3 years ago

I am in a weird scenario were I am forced to use torch 1.3.1 (due to hardware see: https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/). I read from the pytorch docs that it's corresponding version of torchvision is 0.4.1 (https://pypi.org/project/torchvision/):

Installation

We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.

Installation
We recommend Anaconda as Python package management system. Please refer to pytorch.org for the detail of PyTorch (torch) installation. The following is the corresponding torchvision versions and supported Python versions.

torch torchvision python
master / nightly  master / nightly    >=3.6
1.5.0 0.6.0   >=3.5
1.4.0 0.5.0   ==2.7, >=3.5, <=3.8
1.3.1 0.4.2   ==2.7, >=3.5, <=3.7
1.3.0 0.4.1   ==2.7, >=3.5, <=3.7
1.2.0 0.4.0   ==2.7, >=3.5, <=3.7
1.1.0 0.3.0   ==2.7, >=3.5, <=3.7
<=1.0.1   0.2.2   ==2.7, >=3.5, <=3.7

but for some reason I have the wrong version of it:

torchvision 0.2.2 pypi_0 pypi

is there a way to install the right version of torchvision?

What I've tried:

First I tried force installing the right version with conda. Conda couldn't find the version of torchvision that I need:

$ conda install torchvision==0.4.2
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - torchvision==0.4.2

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Then I proceeded to try to install it regardless with pip

$ pip install torchvision==0.4.2
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement torchvision==0.4.2 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3)
ERROR: No matching distribution found for torchvision==0.4.2

got an error too.

Is there anything else to try?

I tried but it failed:

conda install torchvision==0.4.2 -c pytorch

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - torchvision==0.4.2

Current channels:

  - https://conda.anaconda.org/pytorch/linux-ppc64le
  - https://conda.anaconda.org/pytorch/noarch
  - https://repo.anaconda.com/pkgs/main/linux-ppc64le
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-ppc64le
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

related:

jayfurmanek commented 3 years ago

That channel (https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/) is for Watson Machine Learning Community Edition. Read more about it here: https://www.ibm.com/support/knowledgecenter/SS5SF7_1.7.0/navigation/welcome.html

WML CE 1.7.0 includes Pytorch 1.3.1 and torchvision 0.4.2.

You do not have the channel enabled properly. Read the docs above in order to properly do that.

Note that there is also an unsupported update channel with some slightly newer packages: https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda-early-access

Anaconda's defaults channel and the official pytorch channel don't have PyTorch packages for ppc64le.

We've also recently released Open-CE. Please visit us at http://github.com/open-ce

Open-CE has recipes for complete AI environments. The recipes support ppc64le and x86_64. They must be built yourself, but scripts to help take care of that.

brando90 commented 3 years ago

issue were discussion is happening: https://github.com/IBM/powerai/issues/268