Problem with nvidia-cudnn-cu116 it cannot install why is it still using this old version in atsetup?

[CRCODE22]

🔴 If you have installed AllTalk in a custom Python environment, I will only be able to provide limited assistance/support. AllTalk draws on a variety of scripts and libraries that are not written or managed by myself, and they may fail, error or give strange results in custom built python environments.

🔴 Please generate a diagnostics report and upload the "diagnostics.log" as this helps me understand your configuration.

https://github.com/erew123/alltalk_tts/tree/main?#-how-to-make-a-diagnostics-report-file

Latest version of Alltalk before making this issue.

I am using it under windows 11 command prompt with python 3.12.

Collecting nvidia-cudnn-cu116 (from nvidia-cudnn-cu11>=9.0.0.312->-r system\requirements\requirements_textgen.txt (line 22)) Downloading nvidia-cudnn-cu116-0.0.1.dev5.tar.gz (4.4 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [17 lines of output] Traceback (most recent call last): File "", line 2, in File "", line 34, in File "K:\text-generation-webui\installer_files\pip-install-jnik0v5d\nvidia-cudnn-cu116_453fec41241d4b31a213d77d57fe6af3\setup.py", line 152, in raise RuntimeError(open("ERROR.txt", "r").read()) RuntimeError: ########################################################################################### The package you are trying to install is only a placeholder project on PyPI.org repository. This package is hosted on NVIDIA Python Package Index.

  This package can be installed as:
  ```
  $ pip install nvidia-pyindex
  $ pip install nvidia-cudnn-cu116
  ```
  ###########################################################################################

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

There was an error installing the requirements. Have you started your Text-gen-webui Python environment with cmd_{yourOS} before running atsetup.bat? Press any key to return to the menu.

Press any key to continue . . .

[erew123]

Hi @CRCODE22

I believe that text-gen-webui is only running/configuring its Python environment for Python 3.11.x at the moment, which is what I validate AllTalk against.

You mention you are using Python 3.12? Did you use text-generation-webui's start_windows.bat to both install and run text-gen-webui?

Also, did you run text-gen-webui's cmd_windows.bat before you ran atsetup.bat?

Thanks

[erew123]

Hi @CRCODE22

I've just double checked and Nvidia have only still got a CUDNN package for 3.11.x of Python (latest release is April 24th 2024) https://pypi.org/project/nvidia-cudnn-cu11/

CUDNN is used by Finetuning and the TTS generators Analysis function. If you dont want to use either of those and you want to use a custom Python environment (3.12) (assuming TGWUI hasn't move to 3.12), then you can remove the nvidia-cudnn-cu11>=9.0.0.312 from the requirements file https://github.com/erew123/alltalk_tts/blob/main/system/requirements/requirements_textgen.txt and it will install through without that portion.

However, if you did want to use Finetuning or the TTS generators Analysis function on a Python 3.12 environment, you would have to manually install and setup the CUDA toolkit (Please see manual instruction on the below link):

https://github.com/erew123/alltalk_tts#-prerequisites-for-fine-tuning-with-nvidia-cuda-development-toolkit-118

Thanks

[CRCODE22]

Hi @CRCODE22

I believe that text-gen-webui is only running/configuring its Python environment for Python 3.11.x at the moment, which is what I validate AllTalk against.

You mention you are using Python 3.12? Did you use text-generation-webui's start_windows.bat to both install and run text-gen-webui?

Also, did you run text-gen-webui's cmd_windows.bat before you ran atsetup.bat?

Thanks

Yes I have the problem is that it will not load CUDA I have an NVIDIA GeForce RTX 4060 Ti 16GB

Current CUDA Version : CUDA is not available I have installed: v11.7 v11.8 v12.1 v12.3

[CRCODE22]

The standalone version works without problems CUDA Deepspeed everything works

@erew123 The problem is the extension with text generation webui there CUDA does not work.

[CRCODE22]

@erew123

[AllTalk Startup] ____ _ [AllTalk Startup] / \ | | | | _| | | | | / | [AllTalk Startup] / \ | | | | |/ ` | | |/ / | | | | \ \ [AllTalk Startup] / | | | | | (| | | < | | | | ) | [AllTalk Startup] // __|| ||_,|||_\ || || |/ [AllTalk Startup] [AllTalk Startup] Config file check : No Updates required [AllTalk Startup] AllTalk startup Mode : Text-Gen-webui mode [AllTalk Startup] WAV file deletion : Disabled [AllTalk Startup] DeepSpeed version : 0.11.2+unknown [AllTalk Startup] Model is available : Checking [AllTalk Startup] Model is available : Checked [AllTalk Startup] Current Python Version : 3.11.7 [AllTalk Startup] Current PyTorch Version: 2.2.1+cpu [AllTalk Startup] Current CUDA Version : CUDA is not available [AllTalk Startup] Current TTS Version : 0.22.0 [AllTalk Startup] Current TTS Version is : Up to date [AllTalk Startup] AllTalk Github updated : 12th May 2024 at 13:00 [AllTalk Startup] TTS Subprocess : Starting up

[erew123]

Hi @CRCODE22

This means that the current Python Pytorch does not have CUDA extensions installed.

Current PyTorch Version: 2.2.1+cpu

If you have built your own custom Python environment and are not using the one built by text-gen-webui, you will need to have installed PyTorch with CUDA.

Just FYI CUDNN and the CUDA development Toolkits are completely separate things from PyTorch's version on CUDA.

I still am not quite clear if you are using TGWUI's start_windows.bat to have built your Python environment and also to start TGWUI. I will go on the assumption you have not, in which case, you would need to install PyTorch with CUDA on your system.

I would like to be clear that I don't support custom Python environments, so the instructions I will give you perform at your own risk. This is because TGWUI sets up a controlled Python environment that I can say I know exactly how it is set up and whats going on within it. Custom environments are so varied they have to all be looked at on a case by case basis, preferably at the system its-self.

To force an installation of Pytorch with CUDA:

1) Start your Python environment.

If you wish to check your version of PyTorch currently installed you can do so with pip show torch and I believe it will not show +cu121 or any +cuXXX, which indicates that PyTorch with CUDA is not installed, hence its not loading CUDA. (This setup is not something AllTalk installs as I don't change the text-gen-webui environment from what is setup by its installer).

2) pip cache purge (to wipe clean the pip cache as it wont download a full copy of Pytorch otherwise) 3) pip uninstall torch torchaudio torchvision (remove old torch without cuda) 4) You would typically run something such as

pip install torch==2.2.1+cu121 torchaudio>=2.2.1+cu121 torchvision --upgrade --force-reinst all --extra-index-url https://download.pytorch.org/whl/cu121

This would force an install PyTorch 2.2.1 (the current supported version of PyTorch for TGWUI) with CUDA version 12.1 (Also the current supported version on TGWUI)

As far as I am aware, later versions of Pytorch are not yet supported with some of TGWUI's model loaders, so I wouldn't suggest a later version of Pytorch, however, here are the full list of versions https://pytorch.org/get-started/previous-versions/

If you suffer dependency conflicts as a result of not using TGWUI's Python environment, you will have to figure out how to resolve those, which can be very complicated.

Assuming all does go well, you can pip show torch which should now show it with cu121, meaning CUDA extensions are loaded onto Pytorch and AllTalk should report the same on startup.

Just to remind, CUDA Development Toolkits and CUDNN are separate from your PyTorch CUDA version and it is fine to use different toolkits with different Pytorch CUDA versions, hence you will need to follow the instruction I linked to earlier, if you wish to use Finetuning or the TTS Generators analyse function.

Thanks

[CRCODE22]

@erew123 I did use start_windows.bat I am using the environment TGWUI creates I think I know what the problem might be I will post if I know for sure. Thank you for the help so far.

[CRCODE22]

@erew123 it works at least without deep speed for now. Thank you for all the help :)

21:46:04-325454 INFO Loading settings from "settings.yaml" 21:46:04-335492 INFO Loading the extension "alltalktts" [AllTalk Startup] _ __ [AllTalk Startup] / \ | | | | | | | _ | | / | [AllTalk Startup] / \ | | | | |/ ` | | |/ / | | | | _ \ [AllTalk Startup] / | | | | | (| | | < | | | | ) | [AllTalk Startup] // __|| ||\,|||_\ || |_| |/ [AllTalk Startup] [AllTalk Startup] Config file check : No Updates required [AllTalk Startup] AllTalk startup Mode : Text-Gen-webui mode [AllTalk Startup] WAV file deletion : Disabled [AllTalk Startup] DeepSpeed version : Not Detected [AllTalk Startup] Model is available : Checking [AllTalk Startup] Model is available : Checked [AllTalk Startup] Current Python Version : 3.11.9 [AllTalk Startup] Current PyTorch Version: 2.2.1+cu121 [AllTalk Startup] Current CUDA Version : 12.1 [AllTalk Startup] Current TTS Version : 0.22.0 [AllTalk Startup] Current TTS Version is : Up to date [AllTalk Startup] AllTalk Github updated : 12th May 2024 at 13:00 [AllTalk Startup] TTS Subprocess : Starting up [AllTalk Startup] [AllTalk Startup] AllTalk Settings & Documentation: http://127.0.0.1:7851 [AllTalk Startup] [AllTalk Model] XTTSv2 Local Loading xttsv2_2.0.2 into cuda [AllTalk Startup] Warning TTS Subprocess has NOT started up yet, Will keep trying for 120 seconds maximum. Please wait. [AllTalk Startup] Warning TTS Subprocess has NOT started up yet, Will keep trying for 120 seconds maximum. Please wait. [AllTalk Model] Coqui Public Model License [AllTalk Model] https://coqui.ai/cpml.txt [AllTalk Model] Model Loaded in 14.52 seconds. [AllTalk Model] Ready 21:47:13-614231 INFO Loading the extension "whisper_stt" 21:47:13-626804 INFO Loading the extension "openai" 21:47:13-788176 INFO OpenAI-compatible API URL:

                     http://0.0.0.0:5000

21:47:13-790185 INFO Loading the extension "gallery"

Running on local URL: http://0.0.0.0:7862

[Yaffa16]

the problem might be I will post if I know for sure. Thank you for the help so far.

did you figure out how to solve it? I am having the same issue