Instantiating python environment on traverse

rkube commented 2 years ago
Hi, There seems to be something wrong with the yaml file for the environment on traverse.
(base) [rkube@traverse plasma-python]$ conda env create -f envs/requirements-traverse.yaml 
Collecting package metadata (repodata.json): done
Solving environment: \ 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed                                                                                                                                                                                                                             

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Package sqlite conflicts for:
python[version='>=3.6.8'] -> sqlite[version='>=3.26.0,<4.0a0|>=3.28.0,<4.0a0|>=3.30.1,<4.0a0|>=3.32.3,<4.0a0|>=3.33.0,<4.0a0|>=3.34.0,<4.0a0|>=3.35.5,<4.0a0|>=3.36.0,<4.0a0|>=3.37.0,<4.0a0|>=3.37.1,<4.0a0']
Package libuuid conflicts for:
python[version='>=3.6.8'] -> libuuid[version='>=2.32.1,<3.0a0']
Package ld_impl_linux-ppc64le conflicts for:
python[version='>=3.6.8'] -> ld_impl_linux-ppc64le[version='>=2.34|>=2.36.1']
Package cached-property conflicts for:
h5py -> cached-property
Package openssl conflicts for:
python[version='>=3.6.8'] -> openssl[version='>=1.0.2p,<1.0.3a|>=1.1.1a,<1.1.2a|>=1.1.1d,<1.1.2a|>=1.1.1e,<1.1.2a|>=1.1.1f,<1.1.2a|>=1.1.1g,<1.1.2a|>=1.1.1h,<1.1.2a|>=1.1.1i,<1.1.2a|>=1.1.1j,<1.1.2a|>=1.1.1k,<1.1.2a|>=1.1.1l,<1.1.2a|>=1.1.1n,<1.1.2a|>=3.0.0,<4.0a0|>=3.0.2,<4.0a0']
Package xz conflicts for:
python[version='>=3.6.8'] -> xz[version='>=5.2.4,<5.3.0a0|>=5.2.5,<5.3.0a0']
Package libffi conflicts for:
python[version='>=3.6.8'] -> libffi[version='>=3.2.1,<3.3.0a0|>=3.3,<3.4.0a0|>=3.4.2,<3.5.0a0']
Package libzlib conflicts for:
python[version='>=3.6.8'] -> libzlib[version='>=1.2.11,<1.3.0a0']
Package pip conflicts for:
python[version='>=3.6.8'] -> pip
Package libstdcxx-ng conflicts for:
pandas -> libstdcxx-ng[version='>=10.3.0|>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
scipy -> libstdcxx-ng[version='>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
python[version='>=3.6.8'] -> libstdcxx-ng[version='>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
cython -> libstdcxx-ng[version='>=10.3.0|>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
Package configparser conflicts for:
flake8 -> configparser
Package importlib-metadata conflicts for:
flake8 -> importlib-metadata[version='<4.3']
Package bzip2 conflicts for:
python[version='>=3.6.8'] -> bzip2[version='>=1.0.6,<2.0a0|>=1.0.8,<2.0a0']
Package yaml conflicts for:
pyyaml -> yaml[version='>=0.1.7,<0.2.0a0|>=0.2.2,<0.3.0a0|>=0.2.5,<0.3.0a0']
Package apex conflicts for:
pytorch[version='>1.3'] -> apex=0.1.0
Package six conflicts for:
h5py -> six
Package _tflow_select conflicts for:
tensorflow-gpu[version='>=1.3,<2.0.0'] -> _tflow_select==2.1.0[build=gpu*]
Package tzdata conflicts for:
python[version='>=3.6.8'] -> tzdata
Package ncurses conflicts for:
python[version='>=3.6.8'] -> ncurses[version='>=6.1,<7.0.0a0|>=6.2,<6.3.0a0|>=6.2,<7.0.0a0|>=6.3,<7.0a0']
Package openmpi conflicts for:
h5py -> openmpi[version='>=4.0.2,<5.0.0a0|>=4.0.4,<5.0.0a0|>=4.0.5,<5.0.0a0|>=4.1.0,<5.0a0|>=4.1.1,<5.0a0|>=4.1.2,<5.0a0']
Package libblas conflicts for:
scipy -> libblas[version='>=3.8.0,<4.0a0']
Package tk conflicts for:
python[version='>=3.6.8'] -> tk[version='>=8.6.10,<8.7.0a0|>=8.6.11,<8.7.0a0|>=8.6.12,<8.7.0a0|>=8.6.9,<8.7.0a0']
Package typing conflicts for:
flake8 -> typing
Package libgfortran-ng conflicts for:
scipy -> libgfortran-ng[version='>=8,<9.0a0']
Package tensorflow conflicts for:
tensorflow-gpu[version='>=1.3,<2.0.0'] -> tensorflow[version='1.13.1|1.14.0|1.14.0a1|1.15.0|1.15.2|1.15.3|1.15.4|1.15.5']
Package libnsl conflicts for:
python[version='>=3.6.8'] -> libnsl[version='>=2.0.0,<2.1.0a0']
Package zlib conflicts for:
python[version='>=3.6.8'] -> zlib[version='>=1.2.11,<1.3.0a0']
Package numpy conflicts for:
pandas -> numpy[version='>=1.14.5,<2.0a0|>=1.14.6,<2.0a0|>=1.16.4,<2.0a0|>=1.16.5,<2.0a0|>=1.16.6,<2.0a0|>=1.17.5,<2.0a0|>=1.18.1,<2.0a0|>=1.18.4,<2.0a0|>=1.18.5,<2.0a0|>=1.19.2,<2.0a0|>=1.19.4,<2.0a0|>=1.19.5,<2.0a0|>=1.21.4,<2.0a0|>=1.21.5,<2.0a0']
h5py -> numpy[version='>=1.11.3,<2.0a0|>=1.16.5,<2.0a0|>=1.16.6,<2.0a0|>=1.17.5,<2.0a0|>=1.18.5,<2.0a0|>=1.19.2,<2.0a0|>=1.19.5,<2.0a0|>=1.21.2,<2.0a0|>=1.21.4,<2.0a0']
scipy -> numpy[version='>=1.16.4,<2.0a0|>=1.16.5,<2.0a0|>=1.16.6,<2.0a0|>=1.17.5,<2.0a0|>=1.18.1,<2.0a0|>=1.18.5,<2.0a0|>=1.19.2,<2.0a0|>=1.19.4,<2.0a0|>=1.19.5,<2.0a0|>=1.21.4,<2.0a0|>=1.21.5,<2.0a0']
Package unittest2 conflicts for:
h5py -> unittest2
Package wheel conflicts for:
pip -> wheel
Package entrypoints conflicts for:
flake8 -> entrypoints[version='>=0.3.0,<0.4.0']
Package setuptools conflicts for:
pip -> setuptools
pandas -> setuptools[version='<60.0.0']
flake8 -> setuptools[version='>=30.0.0']
Package python conflicts for:
flake8 -> python[version='>=2.7,<2.8.0a0|>=3.5|>=3.6|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
pandas -> python[version='>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0']
scipy -> python[version='>=3.10,<3.11.0a0,>=3.10,<3.11.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.7,<3.8.0a0,>=3.7,<3.8.0a0|>=3.7,<3.8.0a0,>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.8,<3.9.0a0,>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.9,<3.10.0a0,>=3.9,<3.10.0a0',build='*_cpython|*_cpython|*_73_pypy|*_cpython|*_cpython']
cython -> python[version='>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.10,<3.11.0a0,>=3.10,<3.11.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.7,<3.8.0a0,>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.8,<3.9.0a0,>=3.8,<3.9.0a0|>=3.8,<3.9.0a0,>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.9,<3.10.0a0,>=3.9,<3.10.0a0|>=3.9,<3.10.0a0,>=3.9,<3.10.0a0',build='*_73_pypy|*_73_pypy|*_cpython|*_cpython|*_cpython']
h5py -> python[version='>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.10,<3.11.0a0,>=3.10,<3.11.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.7,<3.8.0a0,>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.8,<3.9.0a0,>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.9,<3.10.0a0,>=3.9,<3.10.0a0',build=*_cpython]
pip -> python[version='>=2.7,<2.8.0a0|>=3|>=3.6|>=3.6,<3.7.0a0|>=3.7|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0']
pyparsing -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
pyyaml -> python[version='>=2.7,<2.8.0a0|>=3.10,<3.11.0a0|>=3.10,<3.11.0a0,>=3.10,<3.11.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.7,<3.8.0a0,>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.8,<3.9.0a0,>=3.8,<3.9.0a0|>=3.8,<3.9.0a0,>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.9,<3.10.0a0,>=3.9,<3.10.0a0|>=3.9,<3.10.0a0,>=3.9,<3.10.0a0',build='*_73_pypy|*_73_pypy|*_cpython|*_cpython|*_cpython']
Package libgfortran5 conflicts for:
scipy -> libgfortran5[version='>=8.4.0|>=9.3.0|>=9.4.0']
Package mpich conflicts for:
h5py -> mpich[version='>=3.3.2,<5.0.0a0|>=3.4.1,<5.0a0|>=3.4.2,<5.0a0']
Package pytz conflicts for:
pandas -> pytz[version='>=2017.2']
Package functools32 conflicts for:
flake8 -> functools32
Package python-dateutil conflicts for:
pandas -> python-dateutil[version='>=2.5.*|>=2.6.1|>=2.7.3']
Package readline conflicts for:
python[version='>=3.6.8'] -> readline[version='>=7.0,<8.0a0|>=8.0,<9.0a0|>=8.1,<9.0a0']
Package pytorch-base conflicts for:
pytorch[version='>1.3'] -> pytorch-base=1.3.1
Package hdf5 conflicts for:
h5py -> hdf5[version='>=1.10.1,<1.10.2.0a0|>=1.10.2,<1.10.3.0a0|>=1.10.4,<1.10.5.0a0|>=1.10.5,<1.10.6.0a0|>=1.10.5,<1.10.6.0a0|>=1.10.5,<1.10.6.0a0|>=1.10.6,<1.10.7.0a0|>=1.10.6,<1.10.7.0a0|>=1.10.6,<1.10.7.0a0|>=1.12.1,<1.12.2.0a0|>=1.12.1,<1.12.2.0a0|>=1.12.1,<1.12.2.0a0|>=1.8.20,<1.9.0a0',build='mpi_mpich_*|mpi_openmpi_*|mpi_mpich_*|mpi_mpich_*|mpi_openmpi_*|mpi_openmpi_*']
Package pypy3.8 conflicts for:
pyyaml -> pypy3.8[version='>=7.3.8']
pandas -> pypy3.8[version='>=7.3.9']
python[version='>=3.6.8'] -> pypy3.8[version='7.3.8.*|7.3.9.*']
cython -> pypy3.8[version='>=7.3.8']
Package pypy3.9 conflicts for:
pandas -> pypy3.9[version='>=7.3.9']
pyyaml -> pypy3.9[version='>=7.3.8']
python[version='>=3.6.8'] -> pypy3.9[version='7.3.8.*|7.3.9.*']
cython -> pypy3.9[version='>=7.3.8']
Package pyflakes conflicts for:
flake8 -> pyflakes[version='>=2.1.0,<2.2.0|>=2.2.0,<2.3.0|>=2.3.0,<2.4.0|>=2.4.0,<2.5.0']
Package enum34 conflicts for:
flake8 -> enum34
Package libcblas conflicts for:
scipy -> libcblas[version='>=3.8.0,<4.0a0']
Package python_abi conflicts for:
cython -> python_abi[version='3.10.*|3.6.*|3.7.*|3.8.*|3.9.*|3.6|3.7|3.8|3.9',build='*_pypy38_pp73|*_cp39|*_cp37m|*_cp36m|*_cp310|*_cp38|*_pypy36_pp73|*_pypy37_pp73|*_pypy39_pp73']
pyyaml -> python_abi[version='3.10.*|3.6.*|3.7.*|3.8.*|3.9.*|3.6|3.7|3.8|3.9',build='*_pypy38_pp73|*_cp39|*_cp37m|*_cp36m|*_cp310|*_cp38|*_pypy36_pp73|*_pypy37_pp73|*_pypy39_pp73']
pandas -> python_abi[version='3.10.*|3.6.*|3.7.*|3.8.*|3.9.*|3.6|3.7|3.8|3.9',build='*_pypy38_pp73|*_cp39|*_cp37m|*_cp36m|*_cp310|*_cp38|*_pypy36_pp73|*_pypy37_pp73|*_pypy39_pp73']
h5py -> python_abi[version='3.10.*|3.6.*|3.7.*|3.8.*|3.9.*',build='*_cp39|*_cp37m|*_cp36m|*_cp310|*_cp38']
pip -> python_abi=[build='*_cp37m|*_cp36m|*_cp27mu|*_cp38']
flake8 -> python_abi[version='3.6|3.6.*|3.7.*|3.8.*',build='*_pypy36_pp73|*_cp37m|*_cp36m|*_cp38']
scipy -> python_abi[version='3.10.*|3.6.*|3.7.*|3.8.*|3.9.*|3.6|3.7',build='*_cp39|*_cp37m|*_cp36m|*_cp310|*_cp38|*_pypy36_pp73|*_pypy37_pp73']
python[version='>=3.6.8'] -> python_abi[version='*|*|*|*|3.6|3.7|3.8|3.9',build='*_cp37m|*_cp36m|*_pypy38_pp73|*_pypy37_pp73|*_pypy36_pp73|*_pypy39_pp73|*_cp38|*_cp39']
Package pypy3.6 conflicts for:
python[version='>=3.6.8'] -> pypy3.6[version='7.3.0.*|7.3.1.*|7.3.2.*|7.3.3.*']
pyyaml -> pypy3.6[version='>=7.3.2|>=7.3.3']
cython -> pypy3.6[version='>=7.3.1|>=7.3.2|>=7.3.3']
scipy -> pypy3.6[version='>=7.3.1|>=7.3.2']
pandas -> pypy3.6[version='>=7.3.1|>=7.3.2']
Package pypy3.7 conflicts for:
pandas -> pypy3.7[version='>=7.3.3|>=7.3.4|>=7.3.5|>=7.3.7']
cython -> pypy3.7[version='>=7.3.3|>=7.3.4|>=7.3.5|>=7.3.7']
pyyaml -> pypy3.7[version='>=7.3.3|>=7.3.5|>=7.3.7']
python[version='>=3.6.8'] -> pypy3.7[version='7.3.3.*|7.3.4.*|7.3.5.*|7.3.7.*']
scipy -> pypy3.7[version='>=7.3.3|>=7.3.4|>=7.3.5|>=7.3.7']
Package mccabe conflicts for:
flake8 -> mccabe[version='>=0.6.0,<0.7.0']
Package torchvision-base conflicts for:
pytorch[version='>1.3'] -> torchvision-base=0.4.2
Package torchtext conflicts for:
pytorch[version='>1.3'] -> torchtext=0.4.0
Package libgcc-ng conflicts for:
scipy -> libgcc-ng[version='>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
cython -> libgcc-ng[version='>=10.3.0|>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
pandas -> libgcc-ng[version='>=10.3.0|>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
pyyaml -> libgcc-ng[version='>=10.3.0|>=7.3.0|>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
h5py -> libgcc-ng[version='>=7.2.0|>=7.3.0|>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
python[version='>=3.6.8'] -> libgcc-ng[version='>=10.3.0|>=8.2.0|>=8.4.0|>=9.3.0|>=9.4.0']
Package _pytorch_select conflicts for:
pytorch[version='>1.3'] -> _pytorch_select=2.0
Package pycodestyle conflicts for:
flake8 -> pycodestyle[version='>=2.5.0,<2.6.0|>=2.6.0,<2.7.0|>=2.7.0,<2.8.0|>=2.8.0,<2.9.0']
Package liblapack conflicts for:
scipy -> liblapack[version='>=3.8.0,<4.0.0a0|>=3.8.0,<4.0a0']
Package mpi4py conflicts for:
h5py -> mpi4py[version='>=3.0']
Note that strict channel priority may have removed packages required for satisfiability.

(base) [rkube@traverse plasma-python]$ module list
Currently Loaded Modulefiles:
 1) cudatoolkit/11.6   2) cudnn/cuda-11.x/8.2.0   3) anaconda/2019.10(default)  
(base) [rkube@traverse plasma-python]$
felker commented 2 years ago
The https://github.com/PPPLDeepLearning/plasma-python/blob/master/docs/PrincetonUTutorial.md is already out of date again.
The tf2 branch is essentially the main branch now; master is kept to TF 1.x for legacy reasons
IBM basically stopped updating https://public.dhe.ibm.com/ibmdl/export/pub/software/server/ibm-ai/conda/#/ and users of IBM AC922 machines like Traverse should use https://github.com/open-ce instead
CUDA 11.6 might be too new for some of the packages, hard to say.
rkube commented 2 years ago
Got it, thanks. I'll try installing into a clean environment from open-ce and see how that works out.
rkube commented 2 years ago
Using open-ce resolves the conflicts.
PPPLDeepLearning / plasma-python

Instantiating python environment on traverse #79
