NVIDIA / tensorflow

An Open Source Machine Learning Framework for Everyone
https://developer.nvidia.com/deep-learning-frameworks
Apache License 2.0
962 stars 144 forks source link

nvidia-pyindex breaks pip #98

Open stauntonjr opened 1 year ago

stauntonjr commented 1 year ago

System information

Describe the problem

pip.conf was altered by nvidia-pyindex and now no pip commands work in any environment.

Provide the exact sequence of commands / steps that you executed before running into the problem

$ conda activate myenv
$ pip install nvidia-pyindex

which makes changes to my pip.conf:

# This file has been autogenerated or modified by NVIDIA PyIndex.
# In case you need to modify your PIP configuration, please be aware that
# some configuration files may have a priority order. Here are the following 
# files that may exists in your machine by order of priority:
#
# [Priority 1] Site level configuration files
#   1. `/home/jrs/anaconda3/envs/rapids-23.06/pip.conf`
#
# [Priority 2] User level configuration files
#   1. `/home/jrs/.config/pip/pip.conf`
#   2. `/home/jrs/.pip/pip.conf`
#
# [Priority 3] Global level configuration files
#   1. `/etc/pip.conf`
#   2. `/etc/xdg/pip/pip.conf`

no-cache-dir = true
extra-index-url = https://pypi.ngc.nvidia.com
trusted-host = pypi.ngc.nvidia.com

And now pip commands do not work:

$ pip --version
Configuration file could not be loaded.
File contains no section headers.
file: '/home/jrs/.config/pip/pip.conf', line: 17
'no-cache-dir = true\n'

This affects all other environments based on my path. I tried unsuccessfully to revert per https://github.com/NVIDIA/tensorflow/issues/19#issuecomment-816253523:

$ nvidia_pyindex uninstall
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Uninstalling NVIDIA Pip Configuration ...
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
==============================================================
Cleaning previous NVIDIA PyIndex Configuration ...
File: `/home/jrs/anaconda3/envs/rapids-23.06/pip.conf`
Previous NVIDIA PyIndex Configuration cleaned with success ...
==============================================================
==============================================================
==============================================================
Cleaning previous NVIDIA PyIndex Configuration ...
File: `/home/jrs/.config/pip/pip.conf`
Previous NVIDIA PyIndex Configuration cleaned with success ...
==============================================================
==============================================================
==============================================================
Cleaning previous NVIDIA PyIndex Configuration ...
File: `/home/jrs/.pip/pip.conf`
Error while removing the old NVIDIA PyINDEX configuration: [Errno 2] No such file or directory: '/home/jrs/.pip/pip.conf'
==============================================================
==============================================================
Cleaning previous NVIDIA PyIndex Configuration ...
File: `/etc/pip.conf`
Error while removing the old NVIDIA PyINDEX configuration: [Errno 2] No such file or directory: '/etc/pip.conf'
==============================================================
==============================================================
Cleaning previous NVIDIA PyIndex Configuration ...
File: `/etc/xdg/pip/pip.conf`
Error while removing the old NVIDIA PyINDEX configuration: [Errno 2] No such file or directory: '/etc/xdg/pip/pip.conf'
==============================================================

It seems it didn't know to look where my actual pip.conf file is, unfortunately. Is this normal? What am I missing? Is it related to WSL? How can I fix my pip? I think I now need to rewrite my pip.conf but I don't know what looked like before it was altered.

Any other info / logs N/A

stauntonjr commented 1 year ago

I deleted pip.conf and am able to use pip everywhere again. I don't know what was in my pip.conf prior to that (I believe I was using it to register a private python package repository for a previous client), but so far I haven't run into any issues.

F0gC10ud commented 1 year ago

:fire: :bomb: This is a dangerous bug imho. My pip.conf figs are all gone. :bomb: :fire:

I am unable to figure out what code is doing it, I dont see anything in this repo that would do it.

It simply overwrites all your pip.conf with its idea of "good" settings. In my case this broke all my servers.

.... Meaning all of the carefully crafted configs that I have are all gone and I need to re-craft the ones that were not automated.

stauntonjr commented 1 year ago

How high does NVIDIA stock price have to go before someone addresses this issue?

The adjustments in my pip.conf were made to work on highly sensitive government projects. Luckily the projects were already over when NVIDIA broke my system. But that doesn't mean it isn't a serious issue that NVIDIA needs to fix immediately.

F0gC10ud commented 1 year ago

I have been looking and I cant find it anywhere here. I also tried to reproduce the issue and cant ( I reinstalled a server from scratch) . But it had just happened to me, so it must have been one of the commands that I ran in the last couple hours. /me still trying to reproduce.

jveitchmichaelis commented 3 months ago

I also can't find where the package info is hosted to raise an issue (@DEKHTIARJonathan is listed as the pypi contact - is this in a private repo?)

One addition to this issue - it's unfortunate that nvidia-pyindex automatically disables the (global) pip cache. I work in a - literally - remote environment with slow satellite internet and it took a while to figure out why pip would re-download packages every single time.

This the command that sets up the new config (and overwrites the old one if it exists):

def create_pip_config_file(filepath, config_filedict):

    try:
        print("Creating dir:", os.path.dirname(filepath))
        os.makedirs(os.path.dirname(filepath))
    except FileExistsError:
        pass

    content = generate_configfile_header(config_filedict)
    with open(filepath, 'w') as _file:
        content += """[global]
no-cache-dir = true
index-url = https://pypi.org/simple
extra-index-url =
{extra_index_urls}
trusted-host =
{trusted_hosts}
"""
        extra_index_urls = list()
        trusted_hosts = list()

        for nvidia_pypi_server in NVIDIA_PyPI_SERVERS:
            extra_index_urls.append(_format_extra_index_url(nvidia_pypi_server))
            trusted_hosts.append(_format_trusted_host(nvidia_pypi_server))

        extra_index_urls[-1] = extra_index_urls[-1].rstrip('\n')
        trusted_hosts[-1] = trusted_hosts[-1].rstrip('\n')

        _file.write(
            content.format(
                extra_index_urls="".join(extra_index_urls),
                trusted_hosts="".join(trusted_hosts),
            )
        )

which tracks with the original comment here. I think most people would be better/safer off just adding the urls to their pip config rather than doing this via a package.