Proliferation of (outdated) nvidia management packages

[h-vetinari]

After I found out about scalene, and that conda-forge isn't packaging it yet, I did my usual packager thing of checking the requirements, and whether they're packaged for conda-forge already.

Looking for nvidia-ml-py in various variants lead me to a bunch of feedstocks that are all doing similar things, most of them outdated or obsolete:

• https://github.com/conda-forge/pynvml-feedstock/ -> https://github.com/gpuopenanalytics/pynvml (kinda up to date, but different version (11.0.0 vs. 11.<driver_version>.<build> of the official version - it's unclear if those are compatible); upstream notes:

  As of version 11.0.0, the NVML-wrappers used in pynvml are identical to those published through nvidia-ml-py.

• https://github.com/conda-forge/py3nvml-feedstock -> https://github.com/fbcotter/py3nvml (unofficial fork of nvidia-ml-py at a time when it didn't support python 3)
• https://github.com/conda-forge/nvidia-ml-feedstock -> https://github.com/nicolargo/nvidia-ml-py3 (dead feedstock for a dead python 3 port).

I think at least the last two should be archived, but that still leaves a gap of not being able to use nvidia-ml-py in the same version as the PyPI upstream. I guess one option would be to repackage the pip wheels?

Now that conda-forge can redistribute stuff from nvidia, I hope that this situation can be cleaned up or at least improved.

CC @jakirkham @leofang @kkraus14

[jakirkham]

PyNVML is maintained by folks at NVIDIA. Would raise any issues related to it on that repo.

AIUI the other libraries tried to fill gaps that PyNVML wasn't filling at the time. IIUC those issues have been resolved. Please see issue ( https://github.com/gpuopenanalytics/pynvml/issues/2 ) for some discussion on this.

Ideally more libraries would move to PyNVML, but that is an issue outside the scope of conda-forge.

[h-vetinari]

Thanks for the response @jakirkham. Would you know what's the relation of PyNVML with nvidia-ml-py? The latter is still seeing regular updates on PyPI, and it's not clear to me if they are import-compatible between each other? That would be a gap in conda-forge.

Ideally more libraries would move to PyNVML, but that is an issue outside the scope of conda-forge.

I don't think we're completely out of options on this, but first I'd like to know whether I can fill a pip-requirement "nvidia-ml-py==11.450.51" with pynvml from conda-forge, and how to close that gap if not.