NVIDIA / tensorflow

An Open Source Machine Learning Framework for Everyone
https://developer.nvidia.com/deep-learning-frameworks
Apache License 2.0
968 stars 147 forks source link

Install instructions fail, error message simply repeats install instructions. #15

Closed drscotthawley closed 3 years ago

drscotthawley commented 3 years ago

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template

System information

Describe the problem Trying to follow install instructions for NVIDIA tensorflow via pip (& pyindex) posted at https://github.com/NVIDIA/tensorflow#install results in an error message telling me " This package can be installed as:..." (Re-)Running those commands produces the same error message again.

Provide the exact sequence of commands / steps that you executed before running into the problem Please refer to the log below.

Any other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

$ pip install --user nvidia-pyindex
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting nvidia-pyindex
  Downloading nvidia-pyindex-1.0.6.tar.gz (6.7 kB)
Building wheels for collected packages: nvidia-pyindex
  Building wheel for nvidia-pyindex (setup.py) ... done
  Created wheel for nvidia-pyindex: filename=nvidia_pyindex-1.0.6-py3-none-any.whl size=4171 sha256=692df4078194418f4812516403399f2e96373ad780b93c98ce944b5f02efb35d
  Stored in directory: /tmp/pip-ephem-wheel-cache-kpx26e3z/wheels/52/31/c8/db9f8939a8bb1f3500ce81b630604cbfa6e31f82c8f1bd914d
Successfully built nvidia-pyindex
Installing collected packages: nvidia-pyindex
Successfully installed nvidia-pyindex-1.0.6

$ pip install --user nvidia-tensorflow[horovod]
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting nvidia-tensorflow[horovod]
  Downloading nvidia-tensorflow-0.0.1.dev4.tar.gz (3.8 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/shawley/anaconda3/envs/spnet/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yv_vnm57/nvidia-tensorflow/setup.py'"'"'; __file__='"'"'/tmp/pip-install-yv_vnm57/nvidia-tensorflow/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-1hvhhg4h
         cwd: /tmp/pip-install-yv_vnm57/nvidia-tensorflow/
    Complete output (17 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-yv_vnm57/nvidia-tensorflow/setup.py", line 150, in <module>
        raise RuntimeError(open("ERROR.txt", "r").read())
    RuntimeError:
    ###########################################################################################
    The package you are trying to install is only a placeholder project on PyPI.org repository.
    This package is hosted on NVIDIA Python Package Index.

    This package can be installed as:
$ pip install nvidia-pyindex
$ pip install nvidia-tensorflow
```

Please refer to NVIDIA instructions: https://github.com/NVIDIA/tensorflow#install.
###########################################################################################
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.


Ok, so let's copy exactly from `This package can be installed as:`...

$ pip install nvidia-pyindex Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Requirement already satisfied: nvidia-pyindex in /home/shawley/.local/lib/python3.7/site-packages (1.0.6)

$ pip install nvidia-tensorflow Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com Collecting nvidia-tensorflow Downloading nvidia-tensorflow-0.0.1.dev4.tar.gz (3.8 kB) ERROR: Command errored out with exit status 1: command: /home/shawley/anaconda3/envs/spnet/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ayitiyj8/nvidia-tensorflow/setup.py'"'"'; file='"'"'/tmp/pip-install-ayitiyj8/nvidia-tensorflow/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-v8gqo1_p cwd: /tmp/pip-install-ayitiyj8/nvidia-tensorflow/ Complete output (17 lines): Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-ayitiyj8/nvidia-tensorflow/setup.py", line 150, in raise RuntimeError(open("ERROR.txt", "r").read()) RuntimeError: ########################################################################################### The package you are trying to install is only a placeholder project on PyPI.org repository. This package is hosted on NVIDIA Python Package Index.

This package can be installed as:
```
$ pip install nvidia-pyindex
$ pip install nvidia-tensorflow
```

Please refer to NVIDIA instructions: https://github.com/NVIDIA/tensorflow#install.
###########################################################################################
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.


Referring to https://github.com/NVIDIA/tensorflow#install is what originally got me to this point, so... Not sure what to do next.  Any help? 
drscotthawley commented 3 years ago

$ pip --version pip 20.1.1

DEKHTIARJonathan commented 3 years ago

I'm pretty sure the error comes from Anaconda. Can you try the following for me:

pip install --extra-index-url=https://pypi.ngc.nvidia.com --trusted-host pypi.ngc.nvidia.com nvidia-tensorflow

The objective of the package you try nvidia-pyindex and that fails, install the nvidia python package repository to avoid having to write this complex command. So if it doesn't work, you can go around it like that.

If you are using a docker container, can you please send me the docker container name ? I'd like to be able to reproduce it. Thanks

drscotthawley commented 3 years ago

Thanks for your reply. Looks like I get the same error message:

$ pip install --extra-index-url=https://pypi.ngc.nvidia.com --trusted-host pypi.ngc.nvidia.com nvidia-tensorflow
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com, https://pypi.ngc.nvidia.com
Collecting nvidia-tensorflow
  Downloading nvidia-tensorflow-0.0.1.dev4.tar.gz (3.8 kB)
    ERROR: Command errored out with exit status 1:
     command: /home/shawley/anaconda3/envs/spnet/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-vd7fm7lc/nvidia-tensorflow/setup.py'"'"'; __file__='"'"'/tmp/pip-install-vd7fm7lc/nvidia-tensorflow/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-ir6r10an
         cwd: /tmp/pip-install-vd7fm7lc/nvidia-tensorflow/
    Complete output (17 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-vd7fm7lc/nvidia-tensorflow/setup.py", line 150, in <module>
        raise RuntimeError(open("ERROR.txt", "r").read())
    RuntimeError:
    ###########################################################################################
    The package you are trying to install is only a placeholder project on PyPI.org repository.
    This package is hosted on NVIDIA Python Package Index.

    This package can be installed as:
$ pip install nvidia-pyindex
$ pip install nvidia-tensorflow
```

Please refer to NVIDIA instructions: https://github.com/NVIDIA/tensorflow#install.
###########################################################################################
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.



I'm able to install other packages via conda's pip, just not this package. 

I tried deactivating the conda environment and starting a clean virtualenv environment, but am having other, more general, problems with pip supplied in the venv. 
DEKHTIARJonathan commented 3 years ago

EDIT: I just realized you are using Python 3.7. We don't support that. Only Python3.8 from the December release, and Python 3.6 for November and before

So everything is actually normal. PIP complains he can't find a candidate to install because None exists.

drscotthawley commented 3 years ago

Oh.. thanks, wish I'd read this sooner. I just finished installing Python 3.7 so I could avoid conda and get venv+pip working! LOL. Will try 3.8 instead now.

drscotthawley commented 3 years ago

Ok problem now resolved. Had a couple issues along the way that I will post here for future users who may have similar issues:

With python 3.8 & venv/pip I was getting a different error.

$ python3.8 -m venv env_spnet
$ source env_spnet/bin/activate
(env_spnet) $ pip install --user nvidia-pyindex
Exception:
Traceback (most recent call last):
  File "/home/shawley/env_spnet/lib/python3.8/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/shawley/env_spnet/lib/python3.8/site-packages/pip/commands/install.py", line 290, in run
    with self._build_session(options) as session:
  File "/home/shawley/env_spnet/lib/python3.8/site-packages/pip/basecommand.py", line 68, in _build_session
    normalize_path(os.path.join(options.cache_dir, "http"))
  File "/usr/lib/python3.8/posixpath.py", line 76, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not in

The problem here was that the pip installed by default on Ubuntu is ancient (v9.0). But if you bootstrap pip via

curl https://bootstrap.pypa.io/get-pip.py | python3.8 -

Then you can run the install commands, albeit without the --user part.

After that, installation proceeds without errors. Closing this issue.

mahdaneh commented 3 years ago

Hi, thanks for allowing us to know that nvidia-tensorflow is only working with python3.8. Save my time a lot . Following this instruction , I did not see that requirement in the webpage. I would suggest that maybe the webpage can indicate this in the pre-requirement section.

mrjoness commented 2 years ago

Hello,

I am seeing this same issue but neither python 3.8 nor the --extra-index flag seems to fix the problem. The pyindex command worked with no issues. Here is my current conda list for reference:

# Name                    Version                   Build  Channel
ca-certificates           2021.10.8            h5b45459_0    conda-forge
nvidia-pyindex            1.0.9                    pypi_0    pypi
openssl                   3.0.0                h8ffe710_2    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
python                    3.8.12          h900ac77_2_cpython    conda-forge
python_abi                3.8                      2_cp38    conda-forge
setuptools                60.9.3           py38haa244fe_0    conda-forge
sqlite                    3.37.0               h8ffe710_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
DEKHTIARJonathan commented 2 years ago

@mrjoness we do not support conda

mrjoness commented 2 years ago

Sorry I should clarify that I am using the pip install, this was just to show package versions. This is the full error output:

C:\Users\mrjon\Projects\LSS>pip install --extra-index-url=https://pypi.ngc.nvidia.com --trusted-host pypi.ngc.nvidia.com nvidia-tensorflow[horovod]
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com, https://pypi.ngc.nvidia.com
Collecting nvidia-tensorflow[horovod]
  Downloading nvidia-tensorflow-0.0.1.dev5.tar.gz (7.9 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\mrjon\AppData\Local\Temp\pip-install-b_g8l9g9\nvidia-tensorflow_be127eece9114603ae5b578b2891349a\setup.py", line 150, in <module>
          raise RuntimeError(open("ERROR.txt", "r").read())
      RuntimeError:
      ###########################################################################################
      The package you are trying to install is only a placeholder project on PyPI.org repository.
      This package is hosted on NVIDIA Python Package Index.

      This package can be installed as:
  $ pip install nvidia-pyindex
  $ pip install nvidia-tensorflow
  ```

  Please refer to NVIDIA instructions: https://github.com/NVIDIA/tensorflow#install.
  ###########################################################################################

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

rmc135 commented 1 year ago

I have the same issue under Ubuntu 22.04.1 LTS with Python 3.10.6

Trying:

pip install nvidia-pyindex pip install nvidia-tensorflow[horovod]

or

pip install --extra-index-url=https://pypi.ngc.nvidia.com --trusted-host pypi.ngc.nvidia.com nvidia-tensorflow

Both return the error previously shown which advises the package is a pypi.org placeholder and to see the install instructions.

EDIT: I notice that https://pypi.org/simple returns HTML that appears to be a list of packages, whereas https://pypi.ngc.nvidia.com redirects to https://developer.download.nvidia.com/compute/redist/ (which is a blank page)

I'm unsure whether the pypi.org HTML is there simply for the benefit of humans, or that's the actual index data that pip uses. If it's the latter, that may explain why pip cannot find nvidia-tensorflow on pypi.ngc.nvidia.com.

bokenator commented 1 year ago

I have the same issue under Ubuntu 22.04.1 LTS with Python 3.10.6

Trying:

pip install nvidia-pyindex pip install nvidia-tensorflow[horovod]

or

pip install --extra-index-url=https://pypi.ngc.nvidia.com --trusted-host pypi.ngc.nvidia.com nvidia-tensorflow

Both return the error previously shown which advises the package is a pypi.org placeholder and to see the install instructions.

EDIT: I notice that https://pypi.org/simple returns HTML that appears to be a list of packages, whereas https://pypi.ngc.nvidia.com redirects to https://developer.download.nvidia.com/compute/redist/ (which is a blank page)

I'm unsure whether the pypi.org HTML is there simply for the benefit of humans, or that's the actual index data that pip uses. If it's the latter, that may explain why pip cannot find nvidia-tensorflow on pypi.ngc.nvidia.com.

Facing the exact same issue. Any luck getting it to work?

rmc135 commented 1 year ago

My apologies, I forgot to follow this up.

https://developer.download.nvidia.com/compute/redist/nvidia-tensorflow/ shows the list of WHL files.

Based on those filenames, it seems that "Only Python3.8" from an earlier reply is probably meant to be taken literally? :) ie, no support for 3.9 or later.

I set up a virtual environment with Python 3.8.15 and was able to install nvidia-tensorflow 1.15.5+nv22.11

bagustris commented 1 year ago

In my case, I need to upgrade setuptools since it used the system version, not the virtual environment version.

pip install -U setuptools

Then, I am able to install nvidia-tensorflow[horovod] and nvidia-pyindex without problem.

ghost commented 7 months ago

Try to upgrade pip/setuptools/wheel: Ensure that you have the latest versions of pip, setuptools, and wheel installed. You can upgrade them using the following command:

pip install --upgrade pip setuptools wheel