Open folterj opened 10 months ago
I'm running into the same issue, just to confirm that it indeed is an issue.
Hey, I assume this is to be expected - did you try the normal versions first? Or what is your use case?
I have deleted the beta installers since they weren't supposed to be fully public yet - I had just given them to some people and they become outdated pretty quickly :-) I assume they all fail, although I haven't tested in a while.
Hey! The issue is that the normal version doesn't install, I think that's why we've both tried using the beta installer too. Here's the log from right now, trying a few different ways to install the non-beta, first with tensorflow-gpu
which is the one showing on trex.run:
(base) C:\>conda create -n tracking -c trexing trex numpy=1.23 tensorflow-gpu
Retrieving notices: ...working... done
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- tensorflow-gpu
Current channels:
- https://conda.anaconda.org/trexing/win-64
- https://conda.anaconda.org/trexing/noarch
- https://conda.anaconda.org/conda-forge/win-64
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
and here with just tensorflow
:
(base) C:\>conda create -n tracking -c trexing trex numpy=1.23 tensorflow
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: /
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package tk conflicts for:
numpy=1.23 -> pypy3.9[version='>=7.3.9'] -> tk[version='>=8.6.11,<8.7.0a0|>=8.6.12,<8.7.0a0|>=8.6.13,<8.7.0a0']
trex -> python[version='>=3.9,<3.10.0a0'] -> tk[version='>=8.6.11,<8.7.0a0|>=8.6.12,<8.7.0a0|>=8.6.13,<8.7.0a0']
Package zlib conflicts for:
tensorflow -> grpcio[version='>=1.8.6'] -> zlib[version='1.2.*|1.2.11|>=1.2.11,<1.3.0a0|>=1.2.12,<1.3.0a0|1.2.8']
numpy=1.23 -> pypy3.9[version='>=7.3.9'] -> zlib[version='>=1.2.11,<1.3.0a0|>=1.2.12,<1.3.0a0']
Package numpy conflicts for:
numpy=1.23
tensorflow -> numpy[version='>=1.11.0|>=1.12.1|>=1.13.3']
trex -> scikit-learn -> numpy[version='>=1.11.3,<2.0a0|>=1.14.6,<2.0a0|>=1.16.5,<2.0a0|>=1.16.6,<2.0a0|>=1.17.5,<2.0a0|>=1.18.5,<2.0a0|>=1.19.5,<2.0a0|>=1.20.3,<2.0a0|>=1.21.6,<2.0a0|>=1.22.4,<2.0a0|>=1.23.5,<2.0a0|>=1.26.3,<2.0a0|>=1.26.2,<2.0a0|>=1.26.0,<2.0a0|>=1.23.4,<2.0a0|>=1.21.5,<2.0a0|>=1.21.4,<2.0a0|>=1.19.4,<2.0a0|>=1.19.2,<2.0a0|>=1.22.4,<1.28|>=1.26.3,<1.28|>=1.23.5,<1.28|>=1.26.0,<1.28|>=1.21.6,<1.28|>=1.21.6,<1.27|>=1.23.5,<1.27|>=1.20.3,<1.27|>=1.20.3,<1.26|>=1.23.4,<1.26|>=1.21.6,<1.26|>=1.20.3,<1.25|>=1.21.6,<1.25|>=1.21.6,<1.23|>=1.20.3,<1.23|>=1.12.0|>=1.9.1']
trex -> numpy==1.18.5
tensorflow -> tensorflow-base==1.14.0=py36h9f0ad1d_0 -> numpy[version='>=1.12.0|>=1.16.1|>=1.16.1,<2.0.0a0|>=1.9.1']
Package tensorflow conflicts for:
trex -> keras -> tensorflow[version='>=2.2']
trex -> tensorflow=2.6
tensorflow
without numpy=1.23
:
(base) C:\>conda create -n tracking -c trexing trex tensorflow
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: /
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package tensorflow conflicts for:
trex -> tensorflow=2.6
tensorflow
trex -> keras -> tensorflow[version='>=2.2']
and finally with the command from the latest Github release:
(base) C:\>conda create -n track -c trexing trex
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError:
Hey @mooch443! Just to prioritise my tracking in the coming weeks - do you reckon you will have time to look at/sort this one out? I still have the option of running the tracking on my Mac, but it's also my writing/analysis workhorse, so will take me longer to get through the tracking (it then runs nights only). :-)
conda create -n trex_normal -c trexing trex tensorflow-gpu numpy=1.23.1
this works for me. in general the trouble seems to be that the available software in conda repositories has evolved, but unfortunately i hadn't specified the exact numpy version in this older version of the software. specifying 1.23.1 worked for me (on windows).
Still doesn't work for me:
(base) C:\>conda create -n trex_normal -c trexing trex tensorflow-gpu numpy=1.23.1
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- tensorflow-gpu
Current channels:
- https://conda.anaconda.org/trexing/win-64
- https://conda.anaconda.org/trexing/noarch
- https://conda.anaconda.org/conda-forge/win-64
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Ah, I think I found the issue!
How have you installed conda @mooch443 - with Anaconda? When you create a conda env, does it use a different channel by default? I think if you've installed conda with Anaconda, the default channel is anaconda
, whereas installed with miniforge, the default is conda-forge
.
Now it seems that tensorflow-gpu
is on anaconda
, bu NOT on conda-forge
, which could explain the discrepancies. So maybe we should be more explicit and add the anaconda
channel into the one-liner to ensure it works across all conda installations.
This works for me:
conda create -n trexenv -c trexing -c anaconda trex tensorflow-gpu
(and much much faster solving by using pixi
)
So seems like numpy
may not have been the culprit after all 👍
What version of conda do you have? Maybe you should update that? Because in newer versions of conda you'll get a summary of the current channels being used before the error messages. And it still works for me with the exact command. So within the base environment I recommend doing:
conda update --override-channels -c defaults --all
This is good practice anyway. They may change how information is retrieved from channels in new versions. Make sure it actually updates something and then try again. You can also try adding the defaults to the previous command:
conda create -n trex_normal --override-channels -c trexing -c defaults trex tensorflow-gpu numpy=1.23.1
This is just "conda debugging", which is the most boring part of software development. I just merged my current version into the main dev branch btw, trying to find the last couple missing pieces ;) Looking forward to adding actual new features afterward. You can try this, but you might get a version thats bugged depending on when you download:
# windows or linux
conda create -n betarex --override-channels -c trex-beta -c pytorch -c nvidia -c defaults trex
Ah, I think I found the issue! How have you installed conda @mooch443 - with Anaconda? When you create a conda env, does it use a different channel by default? I think if you've installed conda with Anaconda, the default channel is
anaconda
, whereas installed with miniforge, the default isconda-forge
. Now it seems thattensorflow-gpu
is onanaconda
, bu NOT onconda-forge
, which could explain the discrepancies. So maybe we should be more explicit and add theanaconda
channel into the one-liner to ensure it works across all conda installations.
No I have not installed Anaconda - I am using miniconda, so its just the main/defaults channel. But the solution is to override-channels as in my previous message then. Also updating conda might be a good idea, and getting rid of the anaconda part. Nobody needs the GUI :-D
This works for me:
conda create -n trexenv -c trexing -c anaconda trex tensorflow-gpu
This seems wrong. Can you try my command I suggested above (after updating)?
What is it precisely you dislike about the solution I used? I'm not sure I like the idea about using defaults
channel as that seems to vary between installations, and it's terribly implicit - even when I can now see which channels are being used, I have no idea which channel repository is being used as default (you may know this much better than me, am I understanding this correctly?)
I ran the updating, now none of the solutions work anymore:
(base) C:\>conda create -n trex_normal --override-channels -c trexing -c defaults trex tensorflow-gpu numpy=1.23.1
Channels:
- trexing
- defaults
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\mr630\AppData\Local\mambaforge\envs\trex_normal
added / updated specs:
- numpy=1.23.1
- tensorflow-gpu
- trex
The following packages will be downloaded:
package | build
---------------------------|-----------------
_tflow_select-2.1.0 | gpu 3 KB
abseil-cpp-20210324.2 | hd77b12b_0 1.6 MB
absl-py-1.4.0 | py39haa95532_0 180 KB
aiohttp-3.9.3 | py39h2bbff1b_0 660 KB
aiosignal-1.2.0 | pyhd3eb1b0_0 12 KB
astor-0.8.1 | py39haa95532_0 47 KB
astunparse-1.6.3 | py_0 17 KB
async-timeout-4.0.3 | py39haa95532_0 13 KB
attrs-23.1.0 | py39haa95532_0 143 KB
blas-1.0 | mkl 6 KB
blinker-1.6.2 | py39haa95532_0 29 KB
brotli-python-1.0.9 | py39hd77b12b_7 309 KB
cachetools-4.2.2 | pyhd3eb1b0_0 13 KB
certifi-2024.2.2 | py39haa95532_0 160 KB
cffi-1.16.0 | py39h2bbff1b_0 242 KB
click-8.1.7 | py39haa95532_0 164 KB
colorama-0.4.6 | py39haa95532_0 32 KB
cryptography-41.0.3 | py39h3438e0d_0 1.1 MB
cudatoolkit-11.3.1 | h59b6b97_2 545.3 MB
cudnn-8.2.1 | cuda11.3_0 428.9 MB
ffmpeg-4.2.2 | he774522_0 17.6 MB
flatbuffers-2.0.0 | h6c2663c_0 1.4 MB
frozenlist-1.4.0 | py39h2bbff1b_0 46 KB
gast-0.4.0 | pyhd3eb1b0_0 13 KB
giflib-5.2.1 | h8cc25b3_3 88 KB
google-auth-2.6.0 | pyhd3eb1b0_0 83 KB
google-auth-oauthlib-0.4.1 | py_2 20 KB
google-pasta-0.2.0 | pyhd3eb1b0_0 46 KB
grpcio-1.42.0 | py39hc60d5dd_0 1.9 MB
h5py-3.9.0 | py39hfc34f40_0 914 KB
hdf5-1.12.1 | h51c971a_3 11.9 MB
icc_rt-2022.1.0 | h6049295_2 6.5 MB
icu-68.1 | h6c2663c_0 11.0 MB
idna-3.4 | py39haa95532_0 93 KB
importlib-metadata-7.0.1 | py39haa95532_0 41 KB
intel-openmp-2021.4.0 | haa95532_3556 2.2 MB
joblib-1.2.0 | py39haa95532_0 388 KB
jpeg-9e | h2bbff1b_1 320 KB
keras-preprocessing-1.1.2 | pyhd3eb1b0_0 35 KB
libcurl-8.5.0 | h86230a5_0 343 KB
libpng-1.6.39 | h8cc25b3_0 369 KB
libprotobuf-3.17.2 | h23ce68f_1 1.9 MB
libssh2-1.10.0 | hcd4344a_2 236 KB
markdown-3.4.1 | py39haa95532_0 148 KB
markupsafe-2.1.3 | py39h2bbff1b_0 25 KB
mkl-2021.4.0 | haa95532_640 114.9 MB
mkl-service-2.4.0 | py39h2bbff1b_0 51 KB
mkl_fft-1.3.1 | py39h277e83a_0 139 KB
mkl_random-1.2.2 | py39hf11a4ad_0 225 KB
multidict-6.0.4 | py39h2bbff1b_0 50 KB
numpy-1.23.1 | py39h7a0a035_0 10 KB
numpy-base-1.23.1 | py39hca35cd5_0 5.0 MB
oauthlib-3.2.2 | py39haa95532_0 209 KB
openssl-1.1.1w | h2bbff1b_0 5.5 MB
opt_einsum-3.3.0 | pyhd3eb1b0_1 57 KB
packaging-23.2 | py39haa95532_0 148 KB
pip-23.3.1 | py39haa95532_0 2.8 MB
platformdirs-3.10.0 | py39haa95532_0 35 KB
pooch-1.7.0 | py39haa95532_0 84 KB
protobuf-3.17.2 | py39hd77b12b_0 254 KB
pyasn1-0.4.8 | pyhd3eb1b0_0 54 KB
pyasn1-modules-0.2.8 | py_0 72 KB
pyjwt-2.4.0 | py39haa95532_0 38 KB
pyopenssl-23.2.0 | py39haa95532_0 96 KB
pysocks-1.7.1 | py39haa95532_0 55 KB
python-3.9.18 | h6244533_0 19.4 MB
python-flatbuffers-1.12 | pyhd3eb1b0_0 24 KB
requests-2.31.0 | py39haa95532_1 98 KB
requests-oauthlib-1.3.0 | py_0 23 KB
rsa-4.7.2 | pyhd3eb1b0_1 28 KB
scikit-learn-1.3.0 | py39h4ed8f06_1 7.0 MB
scipy-1.10.1 | py39h321e85e_0 18.7 MB
setuptools-68.2.2 | py39haa95532_0 933 KB
six-1.16.0 | pyhd3eb1b0_1 18 KB
snappy-1.1.10 | h6c2663c_1 92 KB
sqlite-3.41.2 | h2bbff1b_0 894 KB
tensorboard-2.6.0 | py_1 4.9 MB
tensorboard-data-server-0.6.1| py39haa95532_0 17 KB
tensorboard-plugin-wit-1.8.1| py39haa95532_0 671 KB
tensorflow-2.6.0 |gpu_py39he88c5ba_0 4 KB
tensorflow-base-2.6.0 |gpu_py39hb3da07e_0 203.8 MB
tensorflow-estimator-2.6.0 | pyh7b7c402_0 267 KB
tensorflow-gpu-2.6.0 | h17022bd_0 3 KB
termcolor-2.1.0 | py39haa95532_0 12 KB
threadpoolctl-2.2.0 | pyh0d69192_0 16 KB
typing_extensions-4.9.0 | py39haa95532_1 54 KB
urllib3-2.1.0 | py39haa95532_1 154 KB
vs2015_runtime-14.27.29016 | h5e58377_2 1007 KB
werkzeug-2.3.8 | py39haa95532_0 345 KB
wheel-0.35.1 | pyhd3eb1b0_0 38 KB
win_inet_pton-1.1.0 | py39haa95532_0 35 KB
wrapt-1.14.1 | py39h2bbff1b_0 49 KB
yarl-1.9.3 | py39h2bbff1b_0 109 KB
zipp-3.17.0 | py39haa95532_0 23 KB
zlib-1.2.13 | h8cc25b3_0 113 KB
------------------------------------------------------------
Total: 1.39 GB
The following NEW packages will be INSTALLED:
_tflow_select pkgs/main/win-64::_tflow_select-2.1.0-gpu
abseil-cpp pkgs/main/win-64::abseil-cpp-20210324.2-hd77b12b_0
absl-py pkgs/main/win-64::absl-py-1.4.0-py39haa95532_0
aiohttp pkgs/main/win-64::aiohttp-3.9.3-py39h2bbff1b_0
aiosignal pkgs/main/noarch::aiosignal-1.2.0-pyhd3eb1b0_0
astor pkgs/main/win-64::astor-0.8.1-py39haa95532_0
astunparse pkgs/main/noarch::astunparse-1.6.3-py_0
async-timeout pkgs/main/win-64::async-timeout-4.0.3-py39haa95532_0
attrs pkgs/main/win-64::attrs-23.1.0-py39haa95532_0
blas pkgs/main/win-64::blas-1.0-mkl
blinker pkgs/main/win-64::blinker-1.6.2-py39haa95532_0
brotli-python pkgs/main/win-64::brotli-python-1.0.9-py39hd77b12b_7
ca-certificates pkgs/main/win-64::ca-certificates-2024.3.11-haa95532_0
cachetools pkgs/main/noarch::cachetools-4.2.2-pyhd3eb1b0_0
certifi pkgs/main/win-64::certifi-2024.2.2-py39haa95532_0
cffi pkgs/main/win-64::cffi-1.16.0-py39h2bbff1b_0
charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0
click pkgs/main/win-64::click-8.1.7-py39haa95532_0
colorama pkgs/main/win-64::colorama-0.4.6-py39haa95532_0
cryptography pkgs/main/win-64::cryptography-41.0.3-py39h3438e0d_0
cudatoolkit pkgs/main/win-64::cudatoolkit-11.3.1-h59b6b97_2
cudnn pkgs/main/win-64::cudnn-8.2.1-cuda11.3_0
ffmpeg pkgs/main/win-64::ffmpeg-4.2.2-he774522_0
flatbuffers pkgs/main/win-64::flatbuffers-2.0.0-h6c2663c_0
frozenlist pkgs/main/win-64::frozenlist-1.4.0-py39h2bbff1b_0
gast pkgs/main/noarch::gast-0.4.0-pyhd3eb1b0_0
giflib pkgs/main/win-64::giflib-5.2.1-h8cc25b3_3
google-auth pkgs/main/noarch::google-auth-2.6.0-pyhd3eb1b0_0
google-auth-oauth~ pkgs/main/noarch::google-auth-oauthlib-0.4.1-py_2
google-pasta pkgs/main/noarch::google-pasta-0.2.0-pyhd3eb1b0_0
grpcio pkgs/main/win-64::grpcio-1.42.0-py39hc60d5dd_0
h5py pkgs/main/win-64::h5py-3.9.0-py39hfc34f40_0
hdf5 pkgs/main/win-64::hdf5-1.12.1-h51c971a_3
icc_rt pkgs/main/win-64::icc_rt-2022.1.0-h6049295_2
icu pkgs/main/win-64::icu-68.1-h6c2663c_0
idna pkgs/main/win-64::idna-3.4-py39haa95532_0
importlib-metadata pkgs/main/win-64::importlib-metadata-7.0.1-py39haa95532_0
intel-openmp pkgs/main/win-64::intel-openmp-2021.4.0-haa95532_3556
joblib pkgs/main/win-64::joblib-1.2.0-py39haa95532_0
jpeg pkgs/main/win-64::jpeg-9e-h2bbff1b_1
keras-preprocessi~ pkgs/main/noarch::keras-preprocessing-1.1.2-pyhd3eb1b0_0
libcurl pkgs/main/win-64::libcurl-8.5.0-h86230a5_0
libpng pkgs/main/win-64::libpng-1.6.39-h8cc25b3_0
libprotobuf pkgs/main/win-64::libprotobuf-3.17.2-h23ce68f_1
libssh2 pkgs/main/win-64::libssh2-1.10.0-hcd4344a_2
markdown pkgs/main/win-64::markdown-3.4.1-py39haa95532_0
markupsafe pkgs/main/win-64::markupsafe-2.1.3-py39h2bbff1b_0
mkl pkgs/main/win-64::mkl-2021.4.0-haa95532_640
mkl-service pkgs/main/win-64::mkl-service-2.4.0-py39h2bbff1b_0
mkl_fft pkgs/main/win-64::mkl_fft-1.3.1-py39h277e83a_0
mkl_random pkgs/main/win-64::mkl_random-1.2.2-py39hf11a4ad_0
multidict pkgs/main/win-64::multidict-6.0.4-py39h2bbff1b_0
numpy pkgs/main/win-64::numpy-1.23.1-py39h7a0a035_0
numpy-base pkgs/main/win-64::numpy-base-1.23.1-py39hca35cd5_0
oauthlib pkgs/main/win-64::oauthlib-3.2.2-py39haa95532_0
openssl pkgs/main/win-64::openssl-1.1.1w-h2bbff1b_0
opt_einsum pkgs/main/noarch::opt_einsum-3.3.0-pyhd3eb1b0_1
packaging pkgs/main/win-64::packaging-23.2-py39haa95532_0
pip pkgs/main/win-64::pip-23.3.1-py39haa95532_0
platformdirs pkgs/main/win-64::platformdirs-3.10.0-py39haa95532_0
pooch pkgs/main/win-64::pooch-1.7.0-py39haa95532_0
protobuf pkgs/main/win-64::protobuf-3.17.2-py39hd77b12b_0
pyasn1 pkgs/main/noarch::pyasn1-0.4.8-pyhd3eb1b0_0
pyasn1-modules pkgs/main/noarch::pyasn1-modules-0.2.8-py_0
pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0
pyjwt pkgs/main/win-64::pyjwt-2.4.0-py39haa95532_0
pyopenssl pkgs/main/win-64::pyopenssl-23.2.0-py39haa95532_0
pysocks pkgs/main/win-64::pysocks-1.7.1-py39haa95532_0
python pkgs/main/win-64::python-3.9.18-h6244533_0
python-flatbuffers pkgs/main/noarch::python-flatbuffers-1.12-pyhd3eb1b0_0
requests pkgs/main/win-64::requests-2.31.0-py39haa95532_1
requests-oauthlib pkgs/main/noarch::requests-oauthlib-1.3.0-py_0
rsa pkgs/main/noarch::rsa-4.7.2-pyhd3eb1b0_1
scikit-learn pkgs/main/win-64::scikit-learn-1.3.0-py39h4ed8f06_1
scipy pkgs/main/win-64::scipy-1.10.1-py39h321e85e_0
setuptools pkgs/main/win-64::setuptools-68.2.2-py39haa95532_0
six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
snappy pkgs/main/win-64::snappy-1.1.10-h6c2663c_1
sqlite pkgs/main/win-64::sqlite-3.41.2-h2bbff1b_0
tensorboard pkgs/main/noarch::tensorboard-2.6.0-py_1
tensorboard-data-~ pkgs/main/win-64::tensorboard-data-server-0.6.1-py39haa95532_0
tensorboard-plugi~ pkgs/main/win-64::tensorboard-plugin-wit-1.8.1-py39haa95532_0
tensorflow pkgs/main/win-64::tensorflow-2.6.0-gpu_py39he88c5ba_0
tensorflow-base pkgs/main/win-64::tensorflow-base-2.6.0-gpu_py39hb3da07e_0
tensorflow-estima~ pkgs/main/noarch::tensorflow-estimator-2.6.0-pyh7b7c402_0
tensorflow-gpu pkgs/main/win-64::tensorflow-gpu-2.6.0-h17022bd_0
termcolor pkgs/main/win-64::termcolor-2.1.0-py39haa95532_0
threadpoolctl pkgs/main/noarch::threadpoolctl-2.2.0-pyh0d69192_0
trex trexing/win-64::trex-1.1.9-g4ce4be1_0
typing_extensions pkgs/main/win-64::typing_extensions-4.9.0-py39haa95532_1
tzdata pkgs/main/noarch::tzdata-2024a-h04d1e81_0
urllib3 pkgs/main/win-64::urllib3-2.1.0-py39haa95532_1
vc pkgs/main/win-64::vc-14.2-h21ff451_1
vs2015_runtime pkgs/main/win-64::vs2015_runtime-14.27.29016-h5e58377_2
werkzeug pkgs/main/win-64::werkzeug-2.3.8-py39haa95532_0
wheel pkgs/main/noarch::wheel-0.35.1-pyhd3eb1b0_0
win_inet_pton pkgs/main/win-64::win_inet_pton-1.1.0-py39haa95532_0
wrapt pkgs/main/win-64::wrapt-1.14.1-py39h2bbff1b_0
yarl pkgs/main/win-64::yarl-1.9.3-py39h2bbff1b_0
zipp pkgs/main/win-64::zipp-3.17.0-py39haa95532_0
zlib pkgs/main/win-64::zlib-1.2.13-h8cc25b3_0
Proceed ([y]/n)? y
cudatoolkit-11.3.1 | 545.3 MB | #############6 | 18%
cudnn-8.2.1 | 428.9 MB | ###############7 | 21%
tensorflow-base-2.6. | 203.8 MB | ############################################# | 59%
mkl-2021.4.0 | 114.9 MB | ########################################################7 | 75%
tensorflow-base-2.6. | 203.8 MB | ###########################################9 | 58%
mkl-2021.4.0 | 114.9 MB | ######################################################9 | 72%
python-3.9.18 | 19.4 MB | ############################################################################ | 100%
scipy-1.10.1 | 18.7 MB | ############################################################################ | 100%
ffmpeg-4.2.2 | 17.6 MB | ############################################################################ | 100%
hdf5-1.12.1 | 11.9 MB | ############################################################################ | 100%
icu-68.1 | 11.0 MB | ############################################################################ | 100%
openssl-1.1.1w | 5.5 MB | ########################################################################1 | 95%
InvalidArchiveError("Error with archive C:\\Users\\mr630\\AppData\\Local\\mambaforge\\pkgs\\tensorflow-base-2.6.0-gpu_py39hb3da07e_0.conda. You probably need to delete and re-download or re-create this file. Message was:\n\nfailed with error: [Errno 2] No such file or directory: 'C:\\\\Users\\\\mr630\\\\AppData\\\\Local\\\\mambaforge\\\\pkgs\\\\tensorflow-base-2.6.0-gpu_py39hb3da07e_0\\\\Lib\\\\site-packages\\\\tensorflow\\\\include\\\\external\\\\cudnn_frontend_archive\\\\_virtual_includes\\\\cudnn_frontend\\\\third_party\\\\cudnn_frontend\\\\include\\\\cudnn_frontend_EngineConfigGenerator.h'")
When I follow along in the pkgs
folder during the installation, tensorflow-base-2.6.0
is being created, but then it suddenly disappears - no clue why. I'm continuing to try some debugging. (Still works with pixi without any issues).
I think no matter what, it would be good to add a section to the docs about this sort of issue with some general things to attempt. Conda channels is a black box to lots of folks, so having some things to try is good I think.
Oh, you're changing from Tensorflow to PyTorch in the new version?! AMAZING!!! So many headaches from that stuff! (that installation works just fine!)
I see, you should clean your caches then I reckon. conda clean -a
could do the trick - but this is probably just because you haven't done this in a while / they did something illegal, like uploading under the same name?
In any case, as far as I understand defaults
is a combination of the largest and best maintained channels. It is a specific set of channels, not just anaconda or whatever is your "default" setting. I cannot find specific documentation on that, but afaik it includes main
(https://anaconda.org/main), r
and free
. The output you provided seems to suggest that it works as intended. Just that the packages in your cache are broken.
Oh, you're changing from Tensorflow to PyTorch in the new version?! AMAZING!!! So many headaches from that stuff! (that installation works just fine!)
I am using both in fact. Not an easy feat. I would like to port all the previous code to pytorch, but why change a running system. The difference is installing it via pip instead of conda (this process is hidden to the end user). It also means you have to use a fix numpy version.
I see, you should clean your caches then I reckon.
conda clean -a
could do the trick - but this is probably just because you haven't done this in a while / they did something illegal, like uploading under the same name?
Absolutely right, I've never done that in fact! Didn't even know that was a thing. However, the issue remains the same afterwards. Maybe tensorflow-base-2.6.0-gpu
is being overridden by tensorflow-gpu
?
EDIT: Installing without tensorflow-gpu
works, then subsequently installing tensorflow-gpu
reveals that it modifies some of the dependencies. I can't quite figure out what's going on, and unfortunately don't have more time for debugging today, but can try again tomorrow. Here's the output of attempting to install tensorflow-gpu
afterwards:
(trex_normal_2) C:\>conda install tensorflow-gpu
Channels:
- conda-forge
- defaults
- trexing
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\mr630\AppData\Local\mambaforge\envs\trex_normal_2
added / updated specs:
- tensorflow-gpu
The following packages will be downloaded:
package | build
---------------------------|-----------------
certifi-2024.2.2 | pyhd8ed1ab_0 157 KB conda-forge
cudatoolkit-11.3.1 | hf2f0253_13 610.8 MB conda-forge
libprotobuf-3.17.2 | h7755175_1 2.3 MB conda-forge
protobuf-3.17.2 | py39h415ef7b_0 263 KB conda-forge
python_abi-3.9 | 2_cp39 4 KB conda-forge
tensorflow-base-2.6.0 |gpu_py39hb3da07e_0 203.8 MB
vs2015_runtime-14.38.33130 | hcb4865c_18 17 KB conda-forge
------------------------------------------------------------
Total: 817.4 MB
The following NEW packages will be INSTALLED:
python_abi conda-forge/win-64::python_abi-3.9-2_cp39
tensorflow-gpu pkgs/main/win-64::tensorflow-gpu-2.6.0-h17022bd_0
ucrt conda-forge/win-64::ucrt-10.0.22621.0-h57928b3_0
vc14_runtime conda-forge/win-64::vc14_runtime-14.38.33130-h82b7239_18
The following packages will be UPDATED:
libprotobuf pkgs/main::libprotobuf-3.14.0-h23ce68~ --> conda-forge::libprotobuf-3.17.2-h7755175_1
protobuf pkgs/main::protobuf-3.14.0-py39hd77b1~ --> conda-forge::protobuf-3.17.2-py39h415ef7b_0
vs2015_runtime pkgs/main::vs2015_runtime-14.27.29016~ --> conda-forge::vs2015_runtime-14.38.33130-hcb4865c_18
The following packages will be SUPERSEDED by a higher-priority channel:
certifi pkgs/main/win-64::certifi-2024.2.2-py~ --> conda-forge/noarch::certifi-2024.2.2-pyhd8ed1ab_0
cudatoolkit pkgs/main::cudatoolkit-11.8.0-hd77b12~ --> conda-forge::cudatoolkit-11.3.1-hf2f0253_13
cudnn pkgs/main::cudnn-8.9.2.26-cuda11_0 --> conda-forge::cudnn-8.2.1.32-h754d62a_0
openssl pkgs/main::openssl-1.1.1w-h2bbff1b_0 --> conda-forge::openssl-1.1.1w-hcfcfb64_0
The following packages will be DOWNGRADED:
_tflow_select 2.3.0-mkl --> 2.1.0-gpu
tensorflow 2.6.0-mkl_py39h31650da_0 --> 2.6.0-gpu_py39he88c5ba_0
tensorflow-base 2.6.0-mkl_py39h9201259_0 --> 2.6.0-gpu_py39hb3da07e_0
Proceed ([y]/n)? y
Downloading and Extracting Packages:
InvalidArchiveError("Error with archive C:\\Users\\mr630\\AppData\\Local\\mambaforge\\pkgs\\tensorflow-base-2.6.0-gpu_py39hb3da07e_0.conda. You probably need to delete and re-download or re-create this file. Message was:\n\nfailed with error: [Errno 2] No such file or directory: 'C:\\\\Users\\\\mr630\\\\AppData\\\\Local\\\\mambaforge\\\\pkgs\\\\tensorflow-base-2.6.0-gpu_py39hb3da07e_0\\\\Lib\\\\site-packages\\\\tensorflow\\\\include\\\\external\\\\cudnn_frontend_archive\\\\_virtual_includes\\\\cudnn_frontend\\\\third_party\\\\cudnn_frontend\\\\include\\\\cudnn_frontend_EngineConfigGenerator.h'")
In any case, as far as I understand
defaults
is a combination of the largest and best maintained channels. It is a specific set of channels, not just anaconda or whatever is your "default" setting. I cannot find specific documentation on that, but afaik it includesmain
(https://anaconda.org/main),r
andfree
. The output you provided seems to suggest that it works as intended. Just that the packages in your cache are broken.
I think at least the r
channel is no longer getting support - I can't remember where I read it, but it also seems the packages are no longer being updated there. I think it just confuses having implicit channels. It's also one of the reasons prefix-dev (who developed mamba and now pixi) have decided on only having explicit channel names. In pixi they default to conda-forge I think, but every channel has to be named.
Just reiterating, I think some of this information would be really good to place in docs - maybe have a Troubleshooting
section inside of Installation
.
(trex_normal_2) C:>conda install tensorflow-gpu Channels:
- conda-forge
- defaults
- trexing
As you can see, conda-forge is injected here again. This is the reason it tries some weird stuff. You shouldn't mix main/conda-forge channels, this usually breaks stuff!
Regarding the tensorflow-gpu, maybe there is generally some stuff going on with your channels. If you can, reinstall conda and use miniconda - or just install an additional miniconda on your system in a different location. I suspect this might resolve issues.
Just reiterating, I think some of this information would be really good to place in docs - maybe have a
Troubleshooting
section inside ofInstallation
.
I also think this is mainly a conda issue right now, but if there will be similar issues with the new version then I'll add them.
In any case, as far as I understand
defaults
is a combination of the largest and best maintained channels. It is a specific set of channels, not just anaconda or whatever is your "default" setting. I cannot find specific documentation on that, but afaik it includesmain
(https://anaconda.org/main),r
andfree
. The output you provided seems to suggest that it works as intended. Just that the packages in your cache are broken.I think at least the
r
channel is no longer getting support - I can't remember where I read it, but it also seems the packages are no longer being updated there. I think it just confuses having implicit channels. It's also one of the reasons prefix-dev (who developed mamba and now pixi) have decided on only having explicit channel names. In pixi they default to conda-forge I think, but every channel has to be named.
Okay, I was just mentioning that for completeness. The r channel does not really do anything here. I compiled the software using the defaults channels, so if you switch to conda-forge when installing it - even if you get the packages that it wants and it installs "successfully" - it won't work properly in my experience. On Windows it might be more tolerant, but on Linux you won't get it working at all in fact.
(trex_normal_2) C:>conda install tensorflow-gpu Channels:
- conda-forge
- defaults
- trexing
As you can see, conda-forge is injected here again. This is the reason it tries some weird stuff. You shouldn't mix main/conda-forge channels, this usually breaks stuff!
Regarding the tensorflow-gpu, maybe there is generally some stuff going on with your channels. If you can, reinstall conda and use miniconda - or just install an additional miniconda on your system in a different location. I suspect this might resolve issues.
This env was installed with your line, so it's injecting conda-forge even with override-channels. And conda-forge
is not part of my defaults:
(base) C:\WINDOWS\system32>conda config --show default_channels
default_channels:
- https://repo.anaconda.com/pkgs/main
- https://repo.anaconda.com/pkgs/r
- https://repo.anaconda.com/pkgs/msys2
But even when conda-forge is not included, the result is the same:
(base) C:\WINDOWS\system32>conda create -n trex_normal --override-channels -c trexing -c defaults trex numpy=1.23.1
Channels:
- trexing
- defaults
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\mr630\AppData\Local\mambaforge\envs\trex_normal
added / updated specs:
- numpy=1.23.1
- trex
The following NEW packages will be INSTALLED:
_tflow_select pkgs/main/win-64::_tflow_select-2.3.0-mkl
abseil-cpp pkgs/main/win-64::abseil-cpp-20210324.2-hd77b12b_0
absl-py pkgs/main/win-64::absl-py-1.4.0-py39haa95532_0
aiohttp pkgs/main/win-64::aiohttp-3.9.3-py39h2bbff1b_0
aiosignal pkgs/main/noarch::aiosignal-1.2.0-pyhd3eb1b0_0
astor pkgs/main/win-64::astor-0.8.1-py39haa95532_0
astunparse pkgs/main/noarch::astunparse-1.6.3-py_0
async-timeout pkgs/main/win-64::async-timeout-4.0.3-py39haa95532_0
attrs pkgs/main/win-64::attrs-23.1.0-py39haa95532_0
blas pkgs/main/win-64::blas-1.0-mkl
blinker pkgs/main/win-64::blinker-1.6.2-py39haa95532_0
brotli-python pkgs/main/win-64::brotli-python-1.0.9-py39hd77b12b_7
ca-certificates pkgs/main/win-64::ca-certificates-2024.3.11-haa95532_0
cachetools pkgs/main/noarch::cachetools-4.2.2-pyhd3eb1b0_0
certifi pkgs/main/win-64::certifi-2024.2.2-py39haa95532_0
cffi pkgs/main/win-64::cffi-1.16.0-py39h2bbff1b_0
charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0
click pkgs/main/win-64::click-8.1.7-py39haa95532_0
colorama pkgs/main/win-64::colorama-0.4.6-py39haa95532_0
cryptography pkgs/main/win-64::cryptography-41.0.3-py39h3438e0d_0
cudatoolkit pkgs/main/win-64::cudatoolkit-11.8.0-hd77b12b_0
cudnn pkgs/main/win-64::cudnn-8.9.2.26-cuda11_0
ffmpeg pkgs/main/win-64::ffmpeg-4.2.2-he774522_0
flatbuffers pkgs/main/win-64::flatbuffers-2.0.0-h6c2663c_0
frozenlist pkgs/main/win-64::frozenlist-1.4.0-py39h2bbff1b_0
gast pkgs/main/noarch::gast-0.4.0-pyhd3eb1b0_0
giflib pkgs/main/win-64::giflib-5.2.1-h8cc25b3_3
google-auth pkgs/main/noarch::google-auth-2.6.0-pyhd3eb1b0_0
google-auth-oauth~ pkgs/main/noarch::google-auth-oauthlib-0.4.1-py_2
google-pasta pkgs/main/noarch::google-pasta-0.2.0-pyhd3eb1b0_0
grpcio pkgs/main/win-64::grpcio-1.42.0-py39hc60d5dd_0
h5py pkgs/main/win-64::h5py-3.9.0-py39hfc34f40_0
hdf5 pkgs/main/win-64::hdf5-1.12.1-h51c971a_3
icc_rt pkgs/main/win-64::icc_rt-2022.1.0-h6049295_2
icu pkgs/main/win-64::icu-68.1-h6c2663c_0
idna pkgs/main/win-64::idna-3.4-py39haa95532_0
importlib-metadata pkgs/main/win-64::importlib-metadata-7.0.1-py39haa95532_0
intel-openmp pkgs/main/win-64::intel-openmp-2021.4.0-haa95532_3556
joblib pkgs/main/win-64::joblib-1.2.0-py39haa95532_0
jpeg pkgs/main/win-64::jpeg-9e-h2bbff1b_1
keras-preprocessi~ pkgs/main/noarch::keras-preprocessing-1.1.2-pyhd3eb1b0_0
libcurl pkgs/main/win-64::libcurl-8.5.0-h86230a5_0
libpng pkgs/main/win-64::libpng-1.6.39-h8cc25b3_0
libprotobuf pkgs/main/win-64::libprotobuf-3.14.0-h23ce68f_0
libssh2 pkgs/main/win-64::libssh2-1.10.0-hcd4344a_2
markdown pkgs/main/win-64::markdown-3.4.1-py39haa95532_0
markupsafe pkgs/main/win-64::markupsafe-2.1.3-py39h2bbff1b_0
mkl pkgs/main/win-64::mkl-2021.4.0-haa95532_640
mkl-service pkgs/main/win-64::mkl-service-2.4.0-py39h2bbff1b_0
mkl_fft pkgs/main/win-64::mkl_fft-1.3.1-py39h277e83a_0
mkl_random pkgs/main/win-64::mkl_random-1.2.2-py39hf11a4ad_0
multidict pkgs/main/win-64::multidict-6.0.4-py39h2bbff1b_0
numpy pkgs/main/win-64::numpy-1.23.1-py39h7a0a035_0
numpy-base pkgs/main/win-64::numpy-base-1.23.1-py39hca35cd5_0
oauthlib pkgs/main/win-64::oauthlib-3.2.2-py39haa95532_0
openssl pkgs/main/win-64::openssl-1.1.1w-h2bbff1b_0
opt_einsum pkgs/main/noarch::opt_einsum-3.3.0-pyhd3eb1b0_1
packaging pkgs/main/win-64::packaging-23.2-py39haa95532_0
pip pkgs/main/win-64::pip-23.3.1-py39haa95532_0
platformdirs pkgs/main/win-64::platformdirs-3.10.0-py39haa95532_0
pooch pkgs/main/win-64::pooch-1.7.0-py39haa95532_0
protobuf pkgs/main/win-64::protobuf-3.14.0-py39hd77b12b_1
pyasn1 pkgs/main/noarch::pyasn1-0.4.8-pyhd3eb1b0_0
pyasn1-modules pkgs/main/noarch::pyasn1-modules-0.2.8-py_0
pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0
pyjwt pkgs/main/win-64::pyjwt-2.4.0-py39haa95532_0
pyopenssl pkgs/main/win-64::pyopenssl-23.2.0-py39haa95532_0
pysocks pkgs/main/win-64::pysocks-1.7.1-py39haa95532_0
python pkgs/main/win-64::python-3.9.18-h6244533_0
python-flatbuffers pkgs/main/noarch::python-flatbuffers-1.12-pyhd3eb1b0_0
requests pkgs/main/win-64::requests-2.31.0-py39haa95532_1
requests-oauthlib pkgs/main/noarch::requests-oauthlib-1.3.0-py_0
rsa pkgs/main/noarch::rsa-4.7.2-pyhd3eb1b0_1
scikit-learn pkgs/main/win-64::scikit-learn-1.3.0-py39h4ed8f06_1
scipy pkgs/main/win-64::scipy-1.10.1-py39h321e85e_0
setuptools pkgs/main/win-64::setuptools-68.2.2-py39haa95532_0
six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
snappy pkgs/main/win-64::snappy-1.1.10-h6c2663c_1
sqlite pkgs/main/win-64::sqlite-3.41.2-h2bbff1b_0
tensorboard pkgs/main/noarch::tensorboard-2.6.0-py_1
tensorboard-data-~ pkgs/main/win-64::tensorboard-data-server-0.6.1-py39haa95532_0
tensorboard-plugi~ pkgs/main/win-64::tensorboard-plugin-wit-1.8.1-py39haa95532_0
tensorflow pkgs/main/win-64::tensorflow-2.6.0-mkl_py39h31650da_0
tensorflow-base pkgs/main/win-64::tensorflow-base-2.6.0-mkl_py39h9201259_0
tensorflow-estima~ pkgs/main/noarch::tensorflow-estimator-2.6.0-pyh7b7c402_0
termcolor pkgs/main/win-64::termcolor-2.1.0-py39haa95532_0
threadpoolctl pkgs/main/noarch::threadpoolctl-2.2.0-pyh0d69192_0
trex trexing/win-64::trex-1.1.9-g4ce4be1_0
typing_extensions pkgs/main/win-64::typing_extensions-4.9.0-py39haa95532_1
tzdata pkgs/main/noarch::tzdata-2024a-h04d1e81_0
urllib3 pkgs/main/win-64::urllib3-2.1.0-py39haa95532_1
vc pkgs/main/win-64::vc-14.2-h21ff451_1
vs2015_runtime pkgs/main/win-64::vs2015_runtime-14.27.29016-h5e58377_2
werkzeug pkgs/main/win-64::werkzeug-2.3.8-py39haa95532_0
wheel pkgs/main/noarch::wheel-0.35.1-pyhd3eb1b0_0
win_inet_pton pkgs/main/win-64::win_inet_pton-1.1.0-py39haa95532_0
wrapt pkgs/main/win-64::wrapt-1.14.1-py39h2bbff1b_0
yarl pkgs/main/win-64::yarl-1.9.3-py39h2bbff1b_0
zipp pkgs/main/win-64::zipp-3.17.0-py39haa95532_0
zlib pkgs/main/win-64::zlib-1.2.13-h8cc25b3_0
Proceed ([y]/n)? y
Downloading and Extracting Packages:
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate trex_normal
#
# To deactivate an active environment, use
#
# $ conda deactivate
(base) C:\WINDOWS\system32>conda activate trex_normal
(trex_normal) C:\WINDOWS\system32>conda install tensorflow-gpu
Channels:
- conda-forge
- defaults
- trexing
Platform: win-64
Collecting package metadata (repodata.json): failed
CondaError: KeyboardInterrupt
Terminate batch job (Y/N)? y
(trex_normal) C:\WINDOWS\system32>conda install --override-channels -c trexing -c defaults tensorflow-gpu
Channels:
- trexing
- defaults
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\mr630\AppData\Local\mambaforge\envs\trex_normal
added / updated specs:
- tensorflow-gpu
The following packages will be downloaded:
package | build
---------------------------|-----------------
tensorflow-base-2.6.0 |gpu_py39hb3da07e_0 203.8 MB
------------------------------------------------------------
Total: 203.8 MB
The following NEW packages will be INSTALLED:
tensorflow-gpu pkgs/main/win-64::tensorflow-gpu-2.6.0-h17022bd_0
The following packages will be UPDATED:
libprotobuf 3.14.0-h23ce68f_0 --> 3.17.2-h23ce68f_1
protobuf 3.14.0-py39hd77b12b_1 --> 3.17.2-py39hd77b12b_0
The following packages will be DOWNGRADED:
_tflow_select 2.3.0-mkl --> 2.1.0-gpu
cudatoolkit 11.8.0-hd77b12b_0 --> 11.3.1-h59b6b97_2
cudnn 8.9.2.26-cuda11_0 --> 8.2.1-cuda11.3_0
tensorflow 2.6.0-mkl_py39h31650da_0 --> 2.6.0-gpu_py39he88c5ba_0
tensorflow-base 2.6.0-mkl_py39h9201259_0 --> 2.6.0-gpu_py39hb3da07e_0
Proceed ([y]/n)? y
Downloading and Extracting Packages:
InvalidArchiveError("Error with archive C:\\Users\\mr630\\AppData\\Local\\mambaforge\\pkgs\\tensorflow-base-2.6.0-gpu_py39hb3da07e_0.conda. You probably need to delete and re-download or re-create this file. Message was:\n\nfailed with error: [Errno 2] No such file or directory: 'C:\\\\Users\\\\mr630\\\\AppData\\\\Local\\\\mambaforge\\\\pkgs\\\\tensorflow-base-2.6.0-gpu_py39hb3da07e_0\\\\Lib\\\\site-packages\\\\tensorflow\\\\include\\\\external\\\\cudnn_frontend_archive\\\\_virtual_includes\\\\cudnn_frontend\\\\third_party\\\\cudnn_frontend\\\\include\\\\cudnn_frontend_EngineConfigGenerator.h'")
I'll try uninstalling conda tomorrow and try again. Thanks for the ping-pong, I'm sure we'll get to the bottom of it!
All Windows installations as described appear to fail for me. OS: Windows 10 64-bit Python: 3.x 64-bit GPU: Nvidia GTX 1050 Ti
I've tried the approaches as documented without success:
The main error appears to be that a suitable version of tensorflow cannot be found. As a side note, according to the tensorflow documentation, Windows GPU is supported up to TF version 2.10, is currently preferred installed via PIP, not using the 'tensorflow-gpu' package as the regular package automatically detects GPU. I have been able to install tensorflow (and pytorch) separately with GPU support. Installing tensorflow separately before installing TREX, results in TREX unable to find tensorflow, even if the expected version is installed (2.8). I've attempted installing various versions of Python 3 (all 64 bit) and tensorflow (with and without -gpu). The Windows beta installer fails on env/version conflicts (see attached file install.log). For completeness (though this is my least favourite method), the compile using CMake approach fails as bellow (I note it appears to use python 3.12):
Another note is that the python and tensorflow versions do not appear to be pinned in the configurations.