Open gandlf-web opened 4 months ago
Tom-Neverwinter
commented
4 months ago "GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported."
lets start by asking which gpu to get the ball rolling and hopefully resolve this issue
westNeighbor
commented
4 months ago install onnxruntime-gpu as this pip install onnxruntime-gpu==1.18.0 --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/
shalevc1098
commented
4 months ago pip install onnxruntime-gpu==1.18.0 --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/
[17:55:32] Load motion_extractor done. live_portrait_wrapper.py:39
Load warping_module done. live_portrait_wrapper.py:42
Load spade_generator done. live_portrait_wrapper.py:45
Load stitching_retargeting_module done. live_portrait_wrapper.py:49
2024-07-16 17:55:32.8802277 [E:onnxruntime:Default, provider_bridge_ort.cc:1744 onnxruntime::TryGetProviderInfo_CUDA] C:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\AI\LivePortrait\venv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"
*************** EP Error ***************
EP Error C:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:866 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.
when using [('CUDAExecutionProvider', {'device_id': 0})]
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
****************************************
2024-07-16 17:55:32.9229832 [E:onnxruntime:Default, provider_bridge_ort.cc:1744 onnxruntime::TryGetProviderInfo_CUDA] C:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1426 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\AI\LivePortrait\venv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"
Traceback (most recent call last):
File "C:\AI\LivePortrait\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 419, in __init__
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "C:\AI\LivePortrait\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: C:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:866 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\AI\LivePortrait\app.py", line 42, in <module>
gradio_pipeline = GradioPipeline(
File "C:\AI\LivePortrait\src\gradio_pipeline.py", line 28, in __init__
super().__init__(inference_cfg, crop_cfg)
File "C:\AI\LivePortrait\src\live_portrait_pipeline.py", line 38, in __init__
self.cropper: Cropper = Cropper(crop_cfg=crop_cfg)
File "C:\AI\LivePortrait\src\utils\cropper.py", line 50, in __init__
self.landmark_runner = LandmarkRunner(
File "C:\AI\LivePortrait\src\utils\landmark_runner.py", line 37, in __init__
self.session = onnxruntime.InferenceSession(
File "C:\AI\LivePortrait\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 432, in __init__
raise fallback_error from e
File "C:\AI\LivePortrait\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 427, in __init__
self._create_inference_session(self._fallback_providers, None)
File "C:\AI\LivePortrait\venv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: C:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:866 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.cuda, cudnn, versions? https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements
shalevc1098
commented
4 months ago 12.1 and they said that 18 should work with any 12 version
westNeighbor
commented
4 months ago Not clear, If you have right versions of torch, onnxruntime, cuda, cudnn, it should work.
shalevc1098
commented
4 months ago westNeighbor
i have the one for 12.1 --extra-index-url https://download.pytorch.org/whl/cu121 torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0
shalevc1098
commented
4 months ago westNeighbor
commented
4 months ago Your torch is right. And your onnxtrumtime finds the CUDA too. No idea what's the reason. I'm using CUDA 12.4, it works. Maybe you can try reinstalling or install different onnxruntime-gpu version to test.
westNeighbor
commented
4 months ago pip list should show torch 2.3.0+cu121 torchaudio 2.3.0+cu121 torchvision 0.18.0+cu121
shalevc1098
commented
4 months ago pip list should show torch 2.3.0+cu121 torchaudio 2.3.0+cu121 torchvision 0.18.0+cu121
shalevc1098
commented
4 months ago i don't want to install cuda 12.4 so i wouldn't accidentally screw other python projects i use
Tom-Neverwinter
commented
4 months ago i don't want to install cuda 12.4 so i wouldn't accidentally screw other python projects i use
pyenv every time. dont contaminate other projects make them independant
shalevc1098
commented
4 months ago i don't want to install cuda 12.4 so i wouldn't accidentally screw other python projects i use
pyenv every time. dont contaminate other projects make them independant
yes but because i have a project that uses cuda 12.1, if i would install cuda 12.4 instead of what i have i don't want to make it to stop working
hunglsxx
commented
3 months ago @shalevc1098 Have you found a solution yet? I'm experiencing a similar issue. Although the application is still running, there is always a notification like the one in the picture.
Tôi không muốn cài đặt cuda 12.4 vì vậy tôi không muốn vô tình làm hỏng các dự án python khác mà tôi sử dụng
pyenv mọi lúc. không làm ô nhiễm các dự án khác, làm cho chúng độc lập
vâng nhưng vì tôi có một dự án sử dụng cuda 12.1, nếu tôi cài đặt cuda 12.4 thay vì cái tôi đang có thì tôi không muốn nó ngừng hoạt động
FurkanGozukara
commented
3 months ago proper python cuda c++ tools installation explained in this tutorial :
Tinaisok
commented
3 months ago 你好，我在使用LivePortrait遇到了如下的问题： D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:891 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.
File "F:\workstation\ComfyUI-aki-v1.3.7\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "F:\workstation\ComfyUI-aki-v1.3.7\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "F:\workstation\ComfyUI-aki-v1.3.7\execution.py", line 66, in map_node_over_list results.append(getattr(obj, func)(input_data_all)) File "F:\workstation\ComfyUI-aki-v1.3.7\custom_nodes\comfyui-liveportrait\nodes\live_portrait.py", line 364, in run live_portrait_pipeline = LivePortraitPipeline( File "F:\workstation\ComfyUI-aki-v1.3.7\custom_nodes\comfyui-liveportrait\nodes\LivePortrait\src\live_portrait_pipeline.py", line 68, in init self.cropper = Cropper(crop_cfg=crop_cfg,landmark_runner_ckpt=landmark_runner_ckpt,insightface_pretrained_weights=insightface_pretrained_weights) File "F:\workstation\ComfyUI-aki-v1.3.7\custom_nodes\comfyui-liveportrait\nodes\LivePortrait\src\utils\cropper.py", line 45, in init self.landmark_runner = LandmarkRunner( File "F:\workstation\ComfyUI-aki-v1.3.7\custom_nodes\comfyui-liveportrait\nodes\LivePortrait\src\utils\landmark_runner.py", line 36, in init self.session = onnxruntime.InferenceSession( File "F:\workstation\ComfyUI-aki-v1.3.7\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 432, in init raise fallback_error from e File "F:\workstation\ComfyUI-aki-v1.3.7\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 427, in init** self._create_inference_session(self._fallback_providers, None) File "F:\workstation\ComfyUI-aki-v1.3.7\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session sess.initialize_session(providers, provider_options, disabled_optimizers)
https://github.com/KwaiVGI/LivePortrait/issues/321 我刚刚解决了一个dll动态链的问题，之前也是和你一样卡在cuda这里，你可以看看我的问题，换个方法可能可以运行成功
thinkingyuan
commented
3 months ago I hava the same problem.
here is the library version detail:
onnxruntime.version=1.18.0 torch.cuda_version=12.1 torch.cuda.is_available()=True
(liveportrait) D:>pip list | grep torch torch 2.3.0+cu121 torchaudio 2.3.0+cu121 torchvision 0.18.0+cu121
os:windows 10
rmasiso
commented
3 months ago Also suffering through this error. PATHs have been set, cuda is available, cudnn is installed (multiple were tried to see if it was a cudnn version that wasn't working); and even with the 1 click windows installation, I get the same error posted in this issue.
I solved it! So... turns out that you don't need to have your torch cuda install MATCH your actual CUDA. (see here: https://discuss.pytorch.org/t/would-pytorch-for-cuda-11-6-work-when-cuda-is-actually-12-0/169569 )
1) So, what I did is: I created a new environment (using Python 10):
D:\AI> C:\Users\rmasiso\AppData\Local\Programs\Python\Python310\python.exe -m venv LP3102)
Then, I activate the environment via LP310\Scripts\activate.bat
3)
I then pip installed torch for cuda 11.8
pip install torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu1184)
I then installed the requirements file:
pip install -r requirements.txtAnd then, I ran the inference.py script and it worked! No CUDNN issues!
Other important information that I'm not sure how much it helped, but I'll add here in case it's useful.
My environment variables were modified and I included the following:
1) I added a CUDNN_HOME variable and pointed it to one of the CUDNN versions I used:
C:\Program Files\NVIDIA\CUDNN\v8.8.0
2) I had the PATH environment variable include the following:
C:\Program Files\NVIDIA\CUDNN\v8.8.0\bin
3) I'm using CUDNN v8.8.0.121 which I labeled as v8.8.0 when I created the folder for it inside C:\Program Files\NVIDIA\CUDNN because on the onnxruntime website they say that cuda 11.8 AND cuda 12.1 can run with any 8.x for version that is 1.18.0 -- which is what is installed via the requirements.txt file (see here: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements). So, CUDNN v8.8.0.121 was a safe bet.
4) I have CUDA 12.1 installed (because it works well with other AI stuff I'm working on) and didn't want to uninstall it to a lower CUDA version. Driver: 531.14.
Another way to get things to work is to update the onnxruntime. Even though the one on the github suggests onnxrun-gpu to be at 1.18.0, I found that using an updated one ( 1.19.0), did work.
1) Install everything correctly (using torch for cuda 12.1 if that's what you have; following the steps on the github)
2) force update the torch install with a newer one
pip3 install --force-reinstall torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
2) update the onnxruntime by force-reinstalling:
pip install --force-reinstall onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/3) downgrade back to the working numpy version:
pip install numpy==1.26.3
4) run py inference.py
[Unfortunately, even when I was able to build the MultiScaleDeformableAttention, the animal model doesn't work for me, but the human one does.]
Ok. I got everything to work, both human and animal models!
This is how you can do it if you have CUDA 12.1 (and hopefully other versions too).
1) Create a new python environment.
[NOTE: Here I used 3.9 because I was experimenting and it seems this only works with Visual Studio python installs?]
[NOTE: to see your python paths in Windows, you can type: py -0p --this is how I identified the exact path for my python 3.9; otherwise if you create an environment with py -m venv your_env_name it will use the default path, which may not be the one you want.]
D:\rmasiso\PROJECTS\AI>"C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\python.exe" -m venv LP39_2
2) Activate your environment
LP39_2\Scripts\activate
3) Install PyTorch
[This is copied straight from this github]
pip install torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 --index-url https://download.pytorch.org/whl/cu121
4) Install MultiScaleDeformableAttention (but not as is instructed in this repo!)
First:
cd LivePortrait (so that you are inside the repo clone)
Second:
Navigate to the folder containing the dependency (from your LivePortrait clone directory):
cd src/utils/dependencies/XPose/models/UniPose/ops
Third:
BUILD only, not install the setup.py script (learned this from here: https://github.com/fundamentalvision/Deformable-DETR/issues/223):
python setup.py build
The full terminal command should look something like this: (LP39_2) D:\rmasiso\PROJECTS\AI\LivePortrait\src\utils\dependencies\XPose\models\UniPose\ops>python setup.py build
This should start running a bunch of stuff, including warnings, etc. The last few lines of the ouput should look something like:
Creating library build\temp.win-amd64-3.9\Release\rmasiso\PROJECTS\AI\LivePortrait\src\utils\dependencies\XPose\models\UniPose\ops\src\cpu\MultiScaleDeformableAttention.cp39-win_amd64.lib and object build\temp.win-amd64-3.9\Release\rmasiso\PROJECTS\AI\LivePortrait\src\utils\dependencies\XPose\models\UniPose\ops\src\cpu\MultiScaleDeformableAttention.cp39-win_amd64.exp
Generating code
Finished generating codeFourth:
While still inside the \ops directory:
pip3 install .
This will install the dependency via pip.
This is what the full command and output looked like for me:
(LP39_2) D:\rmasiso\PROJECTS\AI\LivePortrait\src\utils\dependencies\XPose\models\UniPose\ops>pip3 install .
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Processing d:\rmasiso\projects\ai\liveportrait\src\utils\dependencies\xpose\models\unipose\ops
Preparing metadata (setup.py) ... done
Using legacy 'setup.py install' for MultiScaleDeformableAttention, since package 'wheel' is not installed.
Installing collected packages: MultiScaleDeformableAttention
Running setup.py install for MultiScaleDeformableAttention ... done
Successfully installed MultiScaleDeformableAttention-1.0Fifth:
Check that the dependency is now listed in the environment with:
pip list
Under there, you should see: MultiScaleDeformableAttention 1.0
Sixth:
You can test that it's working by running python test.py while still in the \ops directory. If a few lines run, then you are good to go. I had a GPU memory error at some point, but that's fine. It's working!
Seventh:
Now you can cd ../../../../../../../ all the way back to \LivePortrait
5) Install the requirements.txt
pip install -r requirements.txt
6) Force-reinstall onnxruntime to a newer version 1.19.0
pip install --force-reinstall onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/
7) Downgrade to a working numpy version:
pip install numpy==1.26.3
8) Test inference.py
py inference.py
9) Test the animal version:
python inference_animals.py -s assets/examples/source/s39.jpg -d assets/examples/driving/wink.pkl --driving_multiplier 1.75 --no_flag_stitching
10) Yay!
Everyone reading this probably hates me at this point (because this is a large *!@ post). But, I'm writing this to save other people time. I thought I had figured it out (see Edit 3), but I couldn't reproduce my solution! So, because my Edit 3 above, was a magical thing that I could not reproduce I needed to figure out how I could actually get this to work without having to downgrade to a different CUDA or install yet more things like CUDA 11.8 Toolkit .
Ok, so I've been able to identify two issues and solutions to fix them:
The CUDA_PATH is set but CUDA wasn't able to be loaded
This issue can be resolved in many ways. One of them was as I described in "Edit 1" of this post. You just use an older pytorch version for CUDA 11.8 (if you are using CUDA 12.1 for example, like me) and you should be able to run the inference.py script.
Another solution to this problem (if you want to use pytorch for CUDA 12.1) is to force-reinstall onnxruntime-gpu to 1.19.0. This will still throw some warnings when you run inference.py but, it in this case, you'll be able to run the scripts. The force-reinstall typically re-installs a newer version of numpy which you'll have to downgrade back to 1.26.3 for the scripts to work.
unable to install xpose with CUDA 12.1
There's many reasons why this could fail. You might get an error that says that D:\rmasiso\PROJECTS\AI\LP39_3\lib\site-packages\torch\utils\cpp_extension.py:499: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend. warnings.warn(msg.format('we could not find ninja.')) or if you somehow manage to build and install it, you'll get an issue like this ImportError: DLL load failed while importing MultiScaleDeformableAttention: %1 is not a valid Win32 application.
The easiest fix for this, the absolute easiest fix I have found... is simply to install this from the original repo: git clone this baby: https://github.com/fundamentalvision/Deformable-DETR ; then while your python environment is active (the one you are using for LivePortrait, and one that has a torch version that matches your active CUDA; for me it was CUDA 12.1), you install the "MultiScaleDeformableAttention" dependency.
You can do so with something like this:
git clone https://github.com/fundamentalvision/Deformable-DETR
cd Deformable-DETR\models\ops
python setup.py build
pip install .
py test.pyIf test.py doesn't run then it didn't work and LivePortrait for animal inference won't work either.
I've tested this method (of installing from the original repo) on Python 3.9, Python 3.10, torch 2.31.0-cu121, and torch 2.30.0-cu121 -- so I've gathered some confidence in reproducing this. By the way, I didn't have to download an older version of CUDA Toolkit.
I hope this helps others that have been struggling with this.
你好，我在使用LivePortrait遇到了如下的问题： D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:891 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasnt able to be loaded. Please install the correct version of CUDA andcuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.
File "F:\workstation\ComfyUI-aki-v1.3.7\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "F:\workstation\ComfyUI-aki-v1.3.7\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "F:\workstation\ComfyUI-aki-v1.3.7\execution.py", line 66, in map_node_over_list results.append(getattr(obj, func)(**input_data_all)) File "F:\workstation\ComfyUI-aki-v1.3.7\custom_nodes\comfyui-liveportrait\nodes\live_portrait.py", line 364, in run live_portrait_pipeline = LivePortraitPipeline( File "F:\workstation\ComfyUI-aki-v1.3.7\custom_nodes\comfyui-liveportrait\nodes\LivePortrait\src\live_portrait_pipeline.py", line 68, in init self.cropper = Cropper(crop_cfg=crop_cfg,landmark_runner_ckpt=landmark_runner_ckpt,insightface_pretrained_weights=insightface_pretrained_weights) File "F:\workstation\ComfyUI-aki-v1.3.7\custom_nodes\comfyui-liveportrait\nodes\LivePortrait\src\utils\cropper.py", line 45, in init self.landmark_runner = LandmarkRunner( File "F:\workstation\ComfyUI-aki-v1.3.7\custom_nodes\comfyui-liveportrait\nodes\LivePortrait\src\utils\landmark_runner.py", line 36, in init self.session = onnxruntime.InferenceSession( File "F:\workstation\ComfyUI-aki-v1.3.7\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 432, in init raise fallback_error from e File "F:\workstation\ComfyUI-aki-v1.3.7\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 427, in init self._create_inference_session(self._fallback_providers, None) File "F:\workstation\ComfyUI-aki-v1.3.7\python\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 483, in _create_inference_session sess.initialize_session(providers, provider_options, disabled_optimizers)