start-life commented 1 year ago

I keep getting this error does anyone have a solution? _

raise Exception("Failed in loading audio2pose_checkpoint")

Exception: Failed in loading audio2pose_checkpoint

Sstan2001 commented 1 year ago

I'm getting this error too. Can anyone help? Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 414, in run_predict output = await app.get_blocks().process_api( File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1323, in process_api result = await self.call_function( File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1051, in call_function prediction = await anyio.to_thread.run_sync( File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "C:\AI\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, *args) File "C:\AI\stable-diffusion-webui/extensions/SadTalker\src\gradio_demo.py", line 42, in test self.audio_to_coeff = Audio2Coeff(self.sadtalker_paths, self.device) File "C:\AI\stable-diffusion-webui/extensions/SadTalker\src\test_audio2coeff.py", line 50, in init raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint

vinthony commented 1 year ago

have you downloaded the checkpoints?

sakura1217 commented 1 year ago

你好，我用 mac M1 第一次安装，报同样的错误： Traceback (most recent call last): File "/Users/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 414, in run_predict output = await app.get_blocks().process_api( File "/Users/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api result = await self.call_function( File "/Users/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1051, in call_function prediction = await anyio.to_thread.run_sync( File "/Users/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/Users/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/Users/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, args) File "/Users/AI/stable-diffusion-webui/modules/call_queue.py", line 15, in f res = func(args, **kwargs) File "/Users/AI/stable-diffusion-webui/extensions/SadTalker/src/gradio_demo.py", line 49, in test self.audio_to_coeff = Audio2Coeff(self.sadtalker_paths, self.device) File "/Users/AI/stable-diffusion-webui/extensions/SadTalker/src/test_audio2coeff.py", line 50, in init raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint

sakura1217 commented 1 year ago

我是用这个命令下载的checkpoint

cd SOMEWHERE_YOU_LIKE

bash <(wget -qO- https://raw.githubusercontent.com/Winfredy/SadTalker/main/scripts/download_models.sh)

start-life commented 1 year ago

I downloaded everything according to this guide https://github.com/OpenTalker/SadTalker/blob/main/docs/webui_extension.md I downloaded the barriers and it still doesn't work and it gives the error I also tried to download the project separately and still have the error I would appreciate it if anything will write a solution

igoso commented 1 year ago

you can try to download all checkpoints from this https://raw.githubusercontent.com/Winfredy/SadTalker/main/scripts/download_models.sh, and put them to SADTALKER_CHECKPOINTS 's checkpoints directory.

start-life commented 1 year ago

Many of the barriers in the file encounter a 404 error. If you could fix it or put an up-to-date file, I would appreciate it.

vinthony commented 1 year ago

You only need to download the files from https://github.com/OpenTalker/SadTalker/releases/tag/v0.0.2-rc. Can you show the full track of the errors, there will have a dict to show all the paths we need in console log.

start-life commented 1 year ago

I downloaded everything and still have the error it's probably not related to this

sakura1217 commented 1 year ago

you can try to download all checkpoints from this https://raw.githubusercontent.com/Winfredy/SadTalker/main/scripts/download_models.sh, and put them to SADTALKER_CHECKPOINTS 's checkpoints directory.

Download all files. Set the file path to display as this image, and that should solve the problem. 571688039494_ pic_hd

Thank you very much！

Sstan2001 commented 1 year ago

Hi Shadow,

Thanks for responding. I have downloaded the 5 files from https://github.com/OpenTalker/SadTalker/releases/tag/v0.0.2-rc many times and followed the instructions (to unzip the files into a SadTalker directory, create a checkpoints folder and put the 2 model.pth.tar and 2 safetensors there.

One thing I did noticed was when I first ran webui.bat (from SadTalker), it downloaded a number of files. 2 of the downloads (I think it was the first one with a 2.2GB file and the third or fourth one) timed out. The rest completed ok. After this compilation, webui.at no longer works -

Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] Commit hash: baf6946e06249c5af9851c60171692c44ef633e0 Installing torch and torchvision Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113 Collecting torch==1.12.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torch-1.12.1%2Bcu113-cp310-cp310-win_amd64.whl (2143.8 MB) Collecting torchvision==0.13.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torchvision-0.13.1%2Bcu113-cp310-cp310-win_amd64.whl (4.7 MB) Collecting torchaudio==0.12.1 Using cached https://download.pytorch.org/whl/cu113/torchaudio-0.12.1%2Bcu113-cp310-cp310-win_amd64.whl (1.2 MB) Collecting typing-extensions Using cached typing_extensions-4.7.0-py3-none-any.whl (33 kB) Collecting requests Using cached requests-2.31.0-py3-none-any.whl (62 kB) Collecting numpy Using cached numpy-1.25.0-cp310-cp310-win_amd64.whl (15.0 MB) Collecting pillow!=8.3.*,>=5.3.0 Using cached Pillow-9.5.0-cp310-cp310-win_amd64.whl (2.5 MB) Collecting certifi>=2017.4.17 Using cached certifi-2023.5.7-py3-none-any.whl (156 kB) Collecting charset-normalizer<4,>=2 Using cached charset_normalizer-3.1.0-cp310-cp310-win_amd64.whl (97 kB) Collecting idna<4,>=2.5 Using cached https://download.pytorch.org/whl/idna-3.4-py3-none-any.whl (61 kB) Collecting urllib3<3,>=1.21.1 Using cached urllib3-2.0.3-py3-none-any.whl (123 kB) Installing collected packages: urllib3, typing-extensions, pillow, numpy, idna, charset-normalizer, certifi, torch, requests, torchvision, torchaudio Successfully installed certifi-2023.5.7 charset-normalizer-3.1.0 idna-3.4 numpy-1.25.0 pillow-9.5.0 requests-2.31.0 torch-1.12.1+cu113 torchaudio-0.12.1+cu113 torchvision-0.13.1+cu113 typing-extensions-4.7.0 urllib3-2.0.3

[notice] A new release of pip available: 22.3.1 -> 23.1.2 [notice] To update, run: python.exe -m pip install --upgrade pip Installing requirements for SadTalker WebUI (may take longer time in first time) Launching SadTalker Web UI RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\Launcher.py", line 199, in start() File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\Launcher.py", line 192, in start from app import sadtalker_demo File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\app.py", line 3, in from src.gradio_demo import SadTalker File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\gradio_demo.py", line 3, in from src.utils.preprocess import CropAndExtract File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\utils\preprocess.py", line 9, in from src.face3d.util.preprocess import align_img File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\face3d\util\preprocess.py", line 5, in from scipy.io import loadmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io__init.py", line 98, in from .matlab import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab__init__.py", line 45, in from ._mio import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab_mio.py", line 9, in from ._mio4 import MatFile4Reader, MatFile4Writer File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab_mio4.py", line 8, in import scipy.sparse File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\sparse\init__.py", line 267, in from ._csr import * File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\sparse_csr.py", line 10, in from ._sparsetools import (csr_tocsc, csr_tobsr, csr_count_blocks, ImportError: numpy.core.multiarray failed to import

Launch unsuccessful. Exiting.

I unzipped into another SadTalker folder, redownloaded the 4 files into the checkpoints folder and ran webui.bat again. But somehow can't get around this error. Please assist.

Thanks, Stacie

From: Shadow Cun @.> Sent: Thursday, June 29, 2023 5:36 AM To: OpenTalker/SadTalker @.> Cc: Sstan2001 @.>; Comment @.> Subject: Re: [OpenTalker/SadTalker] I keep getting this error does anyone have a solution? _ raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint (Issue #441)

You only need to download the files from https://github.com/OpenTalker/SadTalker/releases/tag/v0.0.2-rc. Can you show the full track of the errors, there will have a dict to show all the paths we need in console log.

— Reply to this email directly, view it on GitHubhttps://github.com/OpenTalker/SadTalker/issues/441#issuecomment-1612457936, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZVSDUZSEDMPY3HH47PWG33XNUH5DANCNFSM6AAAAAAZTFT5V4. You are receiving this because you commented.Message ID: @.***>

Sstan2001 commented 1 year ago

Hi Shadow,

I reinstalled SadTalker using the Main (not dev) branch, re-downgraded gradio from

from 3.32.0 to 3.31.0 and it now works if I run webui.bat from the command prompt within Python 3.10.9 and go into SadTalker using the local URL. I managed to generate a video as per normal.

I tried to do the same thing, running SadTalker from Automatic 1111 and using the SadTalker extension tab. When I click on Generate, I get the same error message: Exception: Failed in loading audio2pose_checkpoint

It looks like this error could be related to the Python/Gradio version.

Rgds, Stacie

From: Stacie Tan @.> Sent: Friday, June 30, 2023 1:24 AM To: OpenTalker/SadTalker @.>; OpenTalker/SadTalker @.> Cc: Comment @.> Subject: Re: [OpenTalker/SadTalker] I keep getting this error does anyone have a solution? _ raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint (Issue #441)

Hi Shadow,

Thanks for responding. I have downloaded the 5 files from https://github.com/OpenTalker/SadTalker/releases/tag/v0.0.2-rc many times and followed the instructions (to unzip the files into a SadTalker directory, create a checkpoints folder and put the 2 model.pth.tar and 2 safetensors there.

One thing I did noticed was when I first ran webui.bat (from SadTalker), it downloaded a number of files. 2 of the downloads (I think it was the first one with a 2.2GB file and the third or fourth one) timed out. The rest completed ok. After this compilation, webui.at no longer works -

Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] Commit hash: baf6946e06249c5af9851c60171692c44ef633e0 Installing torch and torchvision Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113 Collecting torch==1.12.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torch-1.12.1%2Bcu113-cp310-cp310-win_amd64.whl (2143.8 MB) Collecting torchvision==0.13.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torchvision-0.13.1%2Bcu113-cp310-cp310-win_amd64.whl (4.7 MB) Collecting torchaudio==0.12.1 Using cached https://download.pytorch.org/whl/cu113/torchaudio-0.12.1%2Bcu113-cp310-cp310-win_amd64.whl (1.2 MB) Collecting typing-extensions Using cached typing_extensions-4.7.0-py3-none-any.whl (33 kB) Collecting requests Using cached requests-2.31.0-py3-none-any.whl (62 kB) Collecting numpy Using cached numpy-1.25.0-cp310-cp310-win_amd64.whl (15.0 MB) Collecting pillow!=8.3.*,>=5.3.0 Using cached Pillow-9.5.0-cp310-cp310-win_amd64.whl (2.5 MB) Collecting certifi>=2017.4.17 Using cached certifi-2023.5.7-py3-none-any.whl (156 kB) Collecting charset-normalizer<4,>=2 Using cached charset_normalizer-3.1.0-cp310-cp310-win_amd64.whl (97 kB) Collecting idna<4,>=2.5 Using cached https://download.pytorch.org/whl/idna-3.4-py3-none-any.whl (61 kB) Collecting urllib3<3,>=1.21.1 Using cached urllib3-2.0.3-py3-none-any.whl (123 kB) Installing collected packages: urllib3, typing-extensions, pillow, numpy, idna, charset-normalizer, certifi, torch, requests, torchvision, torchaudio Successfully installed certifi-2023.5.7 charset-normalizer-3.1.0 idna-3.4 numpy-1.25.0 pillow-9.5.0 requests-2.31.0 torch-1.12.1+cu113 torchaudio-0.12.1+cu113 torchvision-0.13.1+cu113 typing-extensions-4.7.0 urllib3-2.0.3

[notice] A new release of pip available: 22.3.1 -> 23.1.2 [notice] To update, run: python.exe -m pip install --upgrade pip Installing requirements for SadTalker WebUI (may take longer time in first time) Launching SadTalker Web UI RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\Launcher.py", line 199, in start() File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\Launcher.py", line 192, in start from app import sadtalker_demo File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\app.py", line 3, in from src.gradio_demo import SadTalker File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\gradio_demo.py", line 3, in from src.utils.preprocess import CropAndExtract File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\utils\preprocess.py", line 9, in from src.face3d.util.preprocess import align_img File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\face3d\util\preprocess.py", line 5, in from scipy.io import loadmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io__init.py", line 98, in from .matlab import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab__init__.py", line 45, in from ._mio import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab_mio.py", line 9, in from ._mio4 import MatFile4Reader, MatFile4Writer File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab_mio4.py", line 8, in import scipy.sparse File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\sparse\init__.py", line 267, in from ._csr import * File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\sparse_csr.py", line 10, in from ._sparsetools import (csr_tocsc, csr_tobsr, csr_count_blocks, ImportError: numpy.core.multiarray failed to import

Launch unsuccessful. Exiting.

I unzipped into another SadTalker folder, redownloaded the 4 files into the checkpoints folder and ran webui.bat again. But somehow can't get around this error. Please assist.

Thanks, Stacie

From: Shadow Cun @.> Sent: Thursday, June 29, 2023 5:36 AM To: OpenTalker/SadTalker @.> Cc: Sstan2001 @.>; Comment @.> Subject: Re: [OpenTalker/SadTalker] I keep getting this error does anyone have a solution? _ raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint (Issue #441)

You only need to download the files from https://github.com/OpenTalker/SadTalker/releases/tag/v0.0.2-rc. Can you show the full track of the errors, there will have a dict to show all the paths we need in console log.

— Reply to this email directly, view it on GitHubhttps://github.com/OpenTalker/SadTalker/issues/441#issuecomment-1612457936, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZVSDUZSEDMPY3HH47PWG33XNUH5DANCNFSM6AAAAAAZTFT5V4. You are receiving this because you commented.Message ID: @.***>

Sstan2001 commented 1 year ago

BTW, I tried again, with a standalone version of the dev SadTalker. Same error when running webui.bat:- File "C:\AI\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\sparse_csr.py", line 10, in from ._sparsetools import (csr_tocsc, csr_tobsr, csr_count_blocks, ImportError: numpy.core.multiarray failed to import

Launch unsuccessful. Exiting. Press any key to continue . . .

Rgds, Stacie

From: Stacie Tan @.> Sent: Friday, June 30, 2023 7:50 AM To: OpenTalker/SadTalker @.>; OpenTalker/SadTalker @.> Cc: Comment @.> Subject: Re: [OpenTalker/SadTalker] I keep getting this error does anyone have a solution? _ raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint (Issue #441)

Hi Shadow,

I reinstalled SadTalker using the Main (not dev) branch, re-downgraded gradio from

from 3.32.0 to 3.31.0 and it now works if I run webui.bat from the command prompt within Python 3.10.9 and go into SadTalker using the local URL. I managed to generate a video as per normal.

I tried to do the same thing, running SadTalker from Automatic 1111 and using the SadTalker extension tab. When I click on Generate, I get the same error message: Exception: Failed in loading audio2pose_checkpoint

It looks like this error could be related to the Python/Gradio version.

Rgds, Stacie

From: Stacie Tan @.> Sent: Friday, June 30, 2023 1:24 AM To: OpenTalker/SadTalker @.>; OpenTalker/SadTalker @.> Cc: Comment @.> Subject: Re: [OpenTalker/SadTalker] I keep getting this error does anyone have a solution? _ raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint (Issue #441)

Hi Shadow,

Thanks for responding. I have downloaded the 5 files from https://github.com/OpenTalker/SadTalker/releases/tag/v0.0.2-rc many times and followed the instructions (to unzip the files into a SadTalker directory, create a checkpoints folder and put the 2 model.pth.tar and 2 safetensors there.

One thing I did noticed was when I first ran webui.bat (from SadTalker), it downloaded a number of files. 2 of the downloads (I think it was the first one with a 2.2GB file and the third or fourth one) timed out. The rest completed ok. After this compilation, webui.at no longer works -

Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] Commit hash: baf6946e06249c5af9851c60171692c44ef633e0 Installing torch and torchvision Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113 Collecting torch==1.12.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torch-1.12.1%2Bcu113-cp310-cp310-win_amd64.whl (2143.8 MB) Collecting torchvision==0.13.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torchvision-0.13.1%2Bcu113-cp310-cp310-win_amd64.whl (4.7 MB) Collecting torchaudio==0.12.1 Using cached https://download.pytorch.org/whl/cu113/torchaudio-0.12.1%2Bcu113-cp310-cp310-win_amd64.whl (1.2 MB) Collecting typing-extensions Using cached typing_extensions-4.7.0-py3-none-any.whl (33 kB) Collecting requests Using cached requests-2.31.0-py3-none-any.whl (62 kB) Collecting numpy Using cached numpy-1.25.0-cp310-cp310-win_amd64.whl (15.0 MB) Collecting pillow!=8.3.*,>=5.3.0 Using cached Pillow-9.5.0-cp310-cp310-win_amd64.whl (2.5 MB) Collecting certifi>=2017.4.17 Using cached certifi-2023.5.7-py3-none-any.whl (156 kB) Collecting charset-normalizer<4,>=2 Using cached charset_normalizer-3.1.0-cp310-cp310-win_amd64.whl (97 kB) Collecting idna<4,>=2.5 Using cached https://download.pytorch.org/whl/idna-3.4-py3-none-any.whl (61 kB) Collecting urllib3<3,>=1.21.1 Using cached urllib3-2.0.3-py3-none-any.whl (123 kB) Installing collected packages: urllib3, typing-extensions, pillow, numpy, idna, charset-normalizer, certifi, torch, requests, torchvision, torchaudio Successfully installed certifi-2023.5.7 charset-normalizer-3.1.0 idna-3.4 numpy-1.25.0 pillow-9.5.0 requests-2.31.0 torch-1.12.1+cu113 torchaudio-0.12.1+cu113 torchvision-0.13.1+cu113 typing-extensions-4.7.0 urllib3-2.0.3

[notice] A new release of pip available: 22.3.1 -> 23.1.2 [notice] To update, run: python.exe -m pip install --upgrade pip Installing requirements for SadTalker WebUI (may take longer time in first time) Launching SadTalker Web UI RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\Launcher.py", line 199, in start() File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\Launcher.py", line 192, in start from app import sadtalker_demo File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\app.py", line 3, in from src.gradio_demo import SadTalker File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\gradio_demo.py", line 3, in from src.utils.preprocess import CropAndExtract File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\utils\preprocess.py", line 9, in from src.face3d.util.preprocess import align_img File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\src\face3d\util\preprocess.py", line 5, in from scipy.io import loadmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io__init.py", line 98, in from .matlab import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab__init__.py", line 45, in from ._mio import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab_mio.py", line 9, in from ._mio4 import MatFile4Reader, MatFile4Writer File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\io\matlab_mio4.py", line 8, in import scipy.sparse File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\sparse\init__.py", line 267, in from ._csr import * File "C:\AI\stable-diffusion-webui\extensions\SadTalker-0.0.2-rc\venv\lib\site-packages\scipy\sparse_csr.py", line 10, in from ._sparsetools import (csr_tocsc, csr_tobsr, csr_count_blocks, ImportError: numpy.core.multiarray failed to import

Launch unsuccessful. Exiting.

I unzipped into another SadTalker folder, redownloaded the 4 files into the checkpoints folder and ran webui.bat again. But somehow can't get around this error. Please assist.

Thanks, Stacie

From: Shadow Cun @.> Sent: Thursday, June 29, 2023 5:36 AM To: OpenTalker/SadTalker @.> Cc: Sstan2001 @.>; Comment @.> Subject: Re: [OpenTalker/SadTalker] I keep getting this error does anyone have a solution? _ raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint (Issue #441)

You only need to download the files from https://github.com/OpenTalker/SadTalker/releases/tag/v0.0.2-rc. Can you show the full track of the errors, there will have a dict to show all the paths we need in console log.

— Reply to this email directly, view it on GitHubhttps://github.com/OpenTalker/SadTalker/issues/441#issuecomment-1612457936, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZVSDUZSEDMPY3HH47PWG33XNUH5DANCNFSM6AAAAAAZTFT5V4. You are receiving this because you commented.Message ID: @.***>

vinthony commented 1 year ago

These issues are caused by the version of numpy, just change the scipy version in

https://github.com/OpenTalker/SadTalker//requirements.txt#L9C1-L9C14

by scipy==1.10.1 to scipy.

BlazhennovD commented 1 year ago

Hi all! Can you please tell me how to defeat the error:

"FileNotFoundError: [Errno 2] No such file or directory: '5d2e7e36-9724-4e3b-a565-15469dcb4460.mp4'"

Sstan2001 commented 1 year ago

Hi Shadow,

My scipy version is already at 1.10.1 [cid:d946e30f-8376-47cc-8748-e79bdfcfd8d8]

I tried running SadTalker (dev version) standalone again and have the same error:-

venv "venv\Scripts\Python.exe" Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] Commit hash: baf6946e06249c5af9851c60171692c44ef633e0 Installing torch and torchvision Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113 Collecting torch==1.12.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torch-1.12.1%2Bcu113-cp310-cp310-win_amd64.whl (2143.8 MB) Collecting torchvision==0.13.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torchvision-0.13.1%2Bcu113-cp310-cp310-win_amd64.whl (4.7 MB) Collecting torchaudio==0.12.1 Using cached https://download.pytorch.org/whl/cu113/torchaudio-0.12.1%2Bcu113-cp310-cp310-win_amd64.whl (1.2 MB) Collecting typing-extensions Downloading typing_extensions-4.7.1-py3-none-any.whl (33 kB) Collecting numpy Using cached numpy-1.25.0-cp310-cp310-win_amd64.whl (15.0 MB) Collecting requests Using cached requests-2.31.0-py3-none-any.whl (62 kB) Collecting pillow!=8.3.*,>=5.3.0 Downloading Pillow-10.0.0-cp310-cp310-win_amd64.whl (2.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 1.4 MB/s eta 0:00:00 Collecting certifi>=2017.4.17 Using cached certifi-2023.5.7-py3-none-any.whl (156 kB) Collecting charset-normalizer<4,>=2 Using cached charset_normalizer-3.1.0-cp310-cp310-win_amd64.whl (97 kB) Collecting urllib3<3,>=1.21.1 Using cached urllib3-2.0.3-py3-none-any.whl (123 kB) Collecting idna<4,>=2.5 Using cached https://download.pytorch.org/whl/idna-3.4-py3-none-any.whl (61 kB) Installing collected packages: urllib3, typing-extensions, pillow, numpy, idna, charset-normalizer, certifi, torch, requests, torchvision, torchaudio Successfully installed certifi-2023.5.7 charset-normalizer-3.1.0 idna-3.4 numpy-1.25.0 pillow-10.0.0 requests-2.31.0 torch-1.12.1+cu113 torchaudio-0.12.1+cu113 torchvision-0.13.1+cu113 typing-extensions-4.7.1 urllib3-2.0.3

[notice] A new release of pip available: 22.3.1 -> 23.1.2 [notice] To update, run: python.exe -m pip install --upgrade pip Installing requirements for SadTalker WebUI (may take longer time in first time) Launching SadTalker Web UI RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\extensions\SadTalker\Launcher.py", line 199, in start() File "C:\AI\stable-diffusion-webui\extensions\SadTalker\Launcher.py", line 192, in start from app import sadtalker_demo File "C:\AI\stable-diffusion-webui\extensions\SadTalker\app.py", line 3, in from src.gradio_demo import SadTalker File "C:\AI\stable-diffusion-webui\extensions\SadTalker\src\gradio_demo.py", line 3, in from src.utils.preprocess import CropAndExtract File "C:\AI\stable-diffusion-webui\extensions\SadTalker\src\utils\preprocess.py", line 9, in from src.face3d.util.preprocess import align_img File "C:\AI\stable-diffusion-webui\extensions\SadTalker\src\face3d\util\preprocess.py", line 5, in from scipy.io import loadmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\io__init.py", line 98, in from .matlab import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\io\matlab__init__.py", line 45, in from ._mio import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\io\matlab_mio.py", line 9, in from ._mio4 import MatFile4Reader, MatFile4Writer File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\io\matlab_mio4.py", line 8, in import scipy.sparse File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\sparse\init__.py", line 267, in from ._csr import * File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\sparse_csr.py", line 10, in from ._sparsetools import (csr_tocsc, csr_tobsr, csr_count_blocks, ImportError: numpy.core.multiarray failed to import

Launch unsuccessful. Exiting. Press any key to continue . . .

Can you help resolve? Thanks. Stacie

From: Shadow Cun @.> Sent: Monday, July 3, 2023 12:05 PM To: OpenTalker/SadTalker @.> Cc: Sstan2001 @.>; Comment @.> Subject: Re: [OpenTalker/SadTalker] I keep getting this error does anyone have a solution? _ raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint (Issue #441)

These issues are caused by the version of numpy, just change the scipy version in

https://github.com/OpenTalker/SadTalker//requirements.txt#L9C1-L9C14

by scipy==1.10.1 to scipy.

― Reply to this email directly, view it on GitHubhttps://github.com/OpenTalker/SadTalker/issues/441#issuecomment-1618107884, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZVSDU2ZG5EZ345KKFMD253XOKYPXANCNFSM6AAAAAAZTFT5V4. You are receiving this because you commented.Message ID: @.***>

HaSaKiYasuooo commented 1 year ago

您可以尝试从此https://raw.githubusercontent.com/Winfredy/SadTalker/main/scripts/download_models.sh下载所有检查点，并将它们放入SADTALKER_CHECKPOINTS的checkpoints目录中。

下载所有文件。将文件路径设置为显示为该图像，这应该可以解决问题。

非常感谢！

你好，其他的文件如何寻找，

start-life commented 1 year ago

Some of the model links give a 404 error

LianShuaiLong commented 1 year ago

you can try to download all checkpoints from this https://raw.githubusercontent.com/Winfredy/SadTalker/main/scripts/download_models.sh, and put them to SADTALKER_CHECKPOINTS 's checkpoints directory.

Download all files. Set the file path to display as this image, and that should solve the problem.

Thank you very much！

Sorry, it cann`t solve my problem

hyqshr commented 1 year ago

I ran to the same issue on windows. All model that I downloaded in `./checkpoints/' have some weird character at the end which is invisible in file explorer.

Click rename the file and delete that character if there is a space at the end. This solve my issue

bikurifacebook commented 1 year ago

I keep getting this error does anyone have a solution? _
raise Exception("Failed in loading audio2pose_checkpoint")
Exception: Failed in loading audio2pose_checkpoint pip uninstall scipy-1.10.1 pip install scipy-1.11.2

HellKaiser45 commented 1 year ago

You need to download the check points from the bash file: https://raw.githubusercontent.com/Winfredy/SadTalker/main/scripts/download_models.sh If this doesn't work you can try to modify the bash file as follow :

mkdir ./checkpoints  

# lagency download link
curl -L -o ./checkpoints/auido2exp_00300-model.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/auido2exp_00300-model.pth
curl -L -o ./checkpoints/auido2pose_00140-model.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/auido2pose_00140-model.pth
curl -L -o ./checkpoints/epoch_20.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/epoch_20.pth
curl -L -o ./checkpoints/facevid2vid_00189-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/facevid2vid_00189-model.pth.tar
curl -L -o ./checkpoints/shape_predictor_68_face_landmarks.dat https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/shape_predictor_68_face_landmarks.dat
curl -L -o ./checkpoints/wav2lip.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/wav2lip.pth
curl -L -o ./checkpoints/mapping_00229-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/mapping_00229-model.pth.tar
curl -L -o ./checkpoints/mapping_00109-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/mapping_00109-model.pth.tar
curl -L -o ./checkpoints/hub.zip https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/hub.zip
unzip -n ./checkpoints/hub.zip -d ./checkpoints/

#### download the new links.
curl -L -o ./checkpoints/mapping_00109-model.pth.tar https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/mapping_00109-model.pth.tar
curl -L -o ./checkpoints/mapping_00229-model.pth.tar https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/mapping_00229-model.pth.tar
curl -L -o ./checkpoints/SadTalker_V0.0.2_256.safetensors https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/SadTalker_V0.0.2_256.safetensors
curl -L -o ./checkpoints/SadTalker_V0.0.2_512.safetensors https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/SadTalker_V0.0.2_512.safetensors

curl -L -o ./checkpoints/BFM_Fitting.zip https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/BFM_Fitting.zip
unzip -n ./checkpoints/BFM_Fitting.zip -d ./checkpoints/

### enhancer 
mkdir -p ./gfpgan/weights
curl -L -o ./gfpgan/weights/alignment_WFLW_4HG.pth https://github.com/xinntao/facexlib/releases/download/v0.1.0/alignment_WFLW_4HG.pth
curl -L -o ./gfpgan/weights/detection_Resnet50_Final.pth https://github.com/xinntao/facexlib/releases/download/v0.1.0/detection_Resnet50_Final.pth
curl -L -o ./gfpgan/weights/GFPGANv1.4.pth https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth
curl -L -o ./gfpgan/weights/parsing_parsenet.pth https://github.com/xinntao/facexlib/releases/download/v0.2.2/parsing_parsenet.pth

Then, if the directories 'checkpoints' and 'gfpgan' are not directly in the folder SadTalker, copy and paste them inside the 'SadTalker' Folder.

KriszKecskes commented 1 year ago

You need to download the check points from the bash file: https://raw.githubusercontent.com/Winfredy/SadTalker/main/scripts/download_models.sh If this doesn't work you can try to modify the bash file as follow :

mkdir ./checkpoints  

# lagency download link
curl -L -o ./checkpoints/auido2exp_00300-model.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/auido2exp_00300-model.pth
curl -L -o ./checkpoints/auido2pose_00140-model.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/auido2pose_00140-model.pth
curl -L -o ./checkpoints/epoch_20.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/epoch_20.pth
curl -L -o ./checkpoints/facevid2vid_00189-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/facevid2vid_00189-model.pth.tar
curl -L -o ./checkpoints/shape_predictor_68_face_landmarks.dat https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/shape_predictor_68_face_landmarks.dat
curl -L -o ./checkpoints/wav2lip.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/wav2lip.pth
curl -L -o ./checkpoints/mapping_00229-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/mapping_00229-model.pth.tar
curl -L -o ./checkpoints/mapping_00109-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/mapping_00109-model.pth.tar
curl -L -o ./checkpoints/hub.zip https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/hub.zip
unzip -n ./checkpoints/hub.zip -d ./checkpoints/

#### download the new links.
curl -L -o ./checkpoints/mapping_00109-model.pth.tar https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/mapping_00109-model.pth.tar
curl -L -o ./checkpoints/mapping_00229-model.pth.tar https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/mapping_00229-model.pth.tar
curl -L -o ./checkpoints/SadTalker_V0.0.2_256.safetensors https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/SadTalker_V0.0.2_256.safetensors
curl -L -o ./checkpoints/SadTalker_V0.0.2_512.safetensors https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/SadTalker_V0.0.2_512.safetensors

curl -L -o ./checkpoints/BFM_Fitting.zip https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/BFM_Fitting.zip
unzip -n ./checkpoints/BFM_Fitting.zip -d ./checkpoints/

### enhancer 
mkdir -p ./gfpgan/weights
curl -L -o ./gfpgan/weights/alignment_WFLW_4HG.pth https://github.com/xinntao/facexlib/releases/download/v0.1.0/alignment_WFLW_4HG.pth
curl -L -o ./gfpgan/weights/detection_Resnet50_Final.pth https://github.com/xinntao/facexlib/releases/download/v0.1.0/detection_Resnet50_Final.pth
curl -L -o ./gfpgan/weights/GFPGANv1.4.pth https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth
curl -L -o ./gfpgan/weights/parsing_parsenet.pth https://github.com/xinntao/facexlib/releases/download/v0.2.2/parsing_parsenet.pth

Then, if the directories 'checkpoints' and 'gfpgan' are not directly in the folder SadTalker, copy and paste them inside the 'SadTalker' Folder.

Thanks!

cyril23 commented 1 year ago

In the more recent version of https://github.com/OpenTalker/SadTalker/blob/cd4c0465ae0b54a6f85af57f5c65fec9fe23e7f8/scripts/download_models.sh#L5C75-L5C84 the "lagency download link" wget commands are commented out. But the Stable Diffusion extension still needs them apparently?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Source\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Source\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "C:\Source\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "C:\Source\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "C:\Source\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "C:\Source\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Source\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "C:\Source\stable-diffusion-webui\modules\call_queue.py", line 13, in f
    res = func(*args, **kwargs)
  File "C:\Source\stable-diffusion-webui/extensions/SadTalker\src\gradio_demo.py", line 49, in test
    self.audio_to_coeff = Audio2Coeff(self.sadtalker_paths, self.device)
  File "C:\Source\stable-diffusion-webui/extensions/SadTalker\src\test_audio2coeff.py", line 50, in __init__
    raise Exception("Failed in loading audio2pose_checkpoint")
Exception: Failed in loading audio2pose_checkpoint

Anyhow I've managed to fix the error in Windows 11 as follows:

commenting in all the "lagency download link" commands here i.e. removing the # symbol before the commands, e.g. # wget -nc -> wget -nc
choco install wget (in an elevated prompt i.e. with administrator privileges)
cd "C:\Source\stable-diffusion-webui\extensions\SadTalker" (be aware of the "" surrounding the path!) replace C:\Source\stable-diffusion-webui\ with the path wherever you've installed SD!
./download_models.sh
within my main UI webui-user.bat (for me "C:\Source\stable-diffusion-webui\webui-user.bat") I've added the following line above call webui.bat: set SADTALKER_CHECKPOINTS=C:\\Source\\stable-diffusion-webui\\extensions\\SadTalker\\scripts\\checkpoints (be aware of the double \\ or it will not recognize your path!)
Then restart webui-user.bat as usual, and I was able to use the extension without further problems

rucarpe commented 1 year ago

I had the same problem. It was just that I created the folder with the name "checkpoint" instead of "checkpoints"

Jeefrect commented 10 months ago

For some reason my .sh file won't open, it gives an invalid read error. I didn't bother to figure out how to fix it and just made a bat file that does the same thing.

The contents of my bat file that downloads the models:

@echo off

mkdir .\checkpoints

curl -L -o .\checkpoints\auido2exp_00300-model.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/auido2exp_00300-model.pth
curl -L -o .\checkpoints\auido2pose_00140-model.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/auido2pose_00140-model.pth
curl -L -o .\checkpoints\epoch_20.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/epoch_20.pth
curl -L -o .\checkpoints\facevid2vid_00189-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/facevid2vid_00189-model.pth.tar
curl -L -o .\checkpoints\shape_predictor_68_face_landmarks.dat https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/shape_predictor_68_face_landmarks.dat
curl -L -o .\checkpoints\wav2lip.pth https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/wav2lip.pth
curl -L -o .\checkpoints\mapping_00229-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/mapping_00229-model.pth.tar
curl -L -o .\checkpoints\mapping_00109-model.pth.tar https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/mapping_00109-model.pth.tar
curl -L -o .\checkpoints\hub.zip https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/hub.zip
curl -L -o .\checkpoints\BFM_Fitting.zip https://github.com/Winfredy/SadTalker/releases/download/v0.0.2/BFM_Fitting.zip

unzip -n .\checkpoints\hub.zip -d .\checkpoints\

curl -L -o .\checkpoints\mapping_00109-model.pth.tar https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/mapping_00109-model.pth.tar
curl -L -o .\checkpoints\mapping_00229-model.pth.tar https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/mapping_00229-model.pth.tar
curl -L -o .\checkpoints\SadTalker_V0.0.2_256.safetensors https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/SadTalker_V0.0.2_256.safetensors
curl -L -o .\checkpoints\SadTalker_V0.0.2_512.safetensors https://github.com/OpenTalker/SadTalker/releases/download/v0.0.2-rc/SadTalker_V0.0.2_512.safetensors

unzip -n .\checkpoints\BFM_Fitting.zip -d .\checkpoints\

mkdir .\gfpgan\weights

curl -L -o .\gfpgan\weights\alignment_WFLW_4HG.pth https://github.com/xinntao/facexlib/releases/download/v0.1.0/alignment_WFLW_4HG.pth
curl -L -o .\gfpgan\weights\detection_Resnet50_Final.pth https://github.com/xinntao/facexlib/releases/download/v0.1.0/detection_Resnet50_Final.pth
curl -L -o .\gfpgan\weights\GFPGANv1.4.pth https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.4.pth
curl -L -o .\gfpgan\weights\parsing_parsenet.pth https://github.com/xinntao/facexlib/releases/download/v0.2.2/parsing_parsenet.pth

daxige9527 commented 10 months ago

你们仔细检查一下看看是不是这个文件夹的名字打错了，我之前少打了一个S ，研究了一晚上，犯了一个超级低级的错误！

Maikyno commented 9 months ago

I managed to fix the issue by removing "*" symbol which was for some reason added to the end of file names of checkpoints why downloading the data via download script. Hope it may help someone

Full6bClass commented 8 months ago

Good afternoon everyone! I have the same problem" raise exception ("Failed to load audio2pose_checkpoint") Exception: Failed to load audio2pose_checkpoint." This is caused by ""FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints\auido2pose_00140-model.pth'"". I'm guessing it's related to this: https://github.com/OpenTalker/SadTalker?tab=readme-ov-file#old-version. This part describes that this model was used in the old version. But I don't see any explanation on how to use the new model.

Full6bClass commented 8 months ago

The following helped me. Downloaded the v0.0.2 rc release from the link - https://github.com/OpenTalker/SadTalker/releases

replace in req.txt (I did it differently, after installing everything, I disabled the update altogether in the launcher lines from 184 to 187) numba==0.59 scipy==1.12 numpy==1.23.4 gradio==3.31.0

replace this: from torchvision.transforms.functional_tensor import rgb_to_grayscale on this: from torchvision.transforms.functional import rgb_to_grayscale

I’m also too poor for a GPU and use a CPU, so I had to change line 868 in the “serialization.py” file (previously there were 671, look for a search if that) map_location: MAP_LOCATION = None to map_location: MAP_LOCATION = 'cpu'. I apologize that there is a lot that doesn’t relate to this topic, but I hope this will help many at a time.

caterzy commented 6 months ago

Hi Shadow, My scipy version is already at 1.10.1 [cid:d946e30f-8376-47cc-8748-e79bdfcfd8d8] I tried running SadTalker (dev version) standalone again and have the same error:- venv "venv\Scripts\Python.exe" Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] Commit hash: baf6946e06249c5af9851c60171692c44ef633e0 Installing torch and torchvision Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113 Collecting torch==1.12.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torch-1.12.1%2Bcu113-cp310-cp310-win_amd64.whl (2143.8 MB) Collecting torchvision==0.13.1+cu113 Using cached https://download.pytorch.org/whl/cu113/torchvision-0.13.1%2Bcu113-cp310-cp310-win_amd64.whl (4.7 MB) Collecting torchaudio==0.12.1 Using cached https://download.pytorch.org/whl/cu113/torchaudio-0.12.1%2Bcu113-cp310-cp310-win_amd64.whl (1.2 MB) Collecting typing-extensions Downloading typing_extensions-4.7.1-py3-none-any.whl (33 kB) Collecting numpy Using cached numpy-1.25.0-cp310-cp310-win_amd64.whl (15.0 MB) Collecting requests Using cached requests-2.31.0-py3-none-any.whl (62 kB) Collecting pillow!=8.3.,>=5.3.0 Downloading Pillow-10.0.0-cp310-cp310-win_amd64.whl (2.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 1.4 MB/s eta 0:00:00 Collecting certifi>=2017.4.17 Using cached certifi-2023.5.7-py3-none-any.whl (156 kB) Collecting charset-normalizer<4,>=2 Using cached charset_normalizer-3.1.0-cp310-cp310-win_amd64.whl (97 kB) Collecting urllib3<3,>=1.21.1 Using cached urllib3-2.0.3-py3-none-any.whl (123 kB) Collecting idna<4,>=2.5 Using cached https://download.pytorch.org/whl/idna-3.4-py3-none-any.whl (61 kB) Installing collected packages: urllib3, typing-extensions, pillow, numpy, idna, charset-normalizer, certifi, torch, requests, torchvision, torchaudio Successfully installed certifi-2023.5.7 charset-normalizer-3.1.0 idna-3.4 numpy-1.25.0 pillow-10.0.0 requests-2.31.0 torch-1.12.1+cu113 torchaudio-0.12.1+cu113 torchvision-0.13.1+cu113 typing-extensions-4.7.1 urllib3-2.0.3 [notice] A new release of pip available: 22.3.1 -> 23.1.2 [notice] To update, run: python.exe -m pip install --upgrade pip Installing requirements for SadTalker WebUI (may take longer time in first time) Launching SadTalker Web UI RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe Traceback (most recent call last): File "C:\AI\stable-diffusion-webui\extensions\SadTalker\Launcher.py", line 199, in start() File "C:\AI\stable-diffusion-webui\extensions\SadTalker\Launcher.py", line 192, in start from app import sadtalker_demo File "C:\AI\stable-diffusion-webui\extensions\SadTalker\app.py", line 3, in from src.gradio_demo import SadTalker File "C:\AI\stable-diffusion-webui\extensions\SadTalker\src\gradio_demo.py", line 3, in from src.utils.preprocess import CropAndExtract File "C:\AI\stable-diffusion-webui\extensions\SadTalker\src\utils\preprocess.py", line 9, in from src.face3d.util.preprocess import align_img File "C:\AI\stable-diffusion-webui\extensions\SadTalker\src\face3d\util\preprocess.py", line 5, in from scipy.io import loadmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\io__init.py", line 98, in from .matlab import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\io\matlab__init__.py", line 45, in from ._mio import loadmat, savemat, whosmat File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\io\matlab_mio.py", line 9, in from ._mio4 import MatFile4Reader, MatFile4Writer File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\io\matlab_mio4.py", line 8, in import scipy.sparse File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\sparse\init__.py", line 267, in from ._csr import File "C:\AI\stable-diffusion-webui\extensions\SadTalker\venv\lib\site-packages\scipy\sparse_csr.py", line 10, in from ._sparsetools import (csr_tocsc, csr_tobsr, csr_countblocks, ImportError: numpy.core.multiarray failed to import Launch unsuccessful. Exiting. Press any key to continue . . . Can you help resolve? Thanks. Stacie … ____ From: Shadow Cun @.> Sent: Monday, July 3, 2023 12:05 PM To: OpenTalker/SadTalker @.> Cc: Sstan2001 @.>; Comment @.> Subject: Re: [OpenTalker/SadTalker] I keep getting this error does anyone have a solution? raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint (Issue #441) These issues are caused by the version of numpy, just change the scipy version in https://github.com/OpenTalker/SadTalker//requirements.txt#L9C1-L9C14 by scipy==1.10.1 to scipy. ― Reply to this email directly, view it on GitHub<#441 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZVSDU2ZG5EZ345KKFMD253XOKYPXANCNFSM6AAAAAAZTFT5V4. You are receiving this because you commented.Message ID: @.***>

python version

lipeng6723 commented 6 months ago

看了你的回复，居然解决了我遇到的问题，https://github.com/OpenTalker/SadTalker/releases/tag/v0.0.2-rc 网址里面前四个文件，第三个第四个解压，第一个第二个不要解压，放在新建的 checkpoints目录下，运行webui.bat 居然就好使了！

Yaqing2023 commented 2 months ago

the download script has ...download/v0.0.2/auido2pose_00140-model.pth ..., notice the file name is auido..., the exception show Exception: Failed in loading audio2pose_checkpoint, notice its loading audio..., so the filename should be auido or audio?

Gustavo189 commented 2 months ago

I got it working fixing this issue, the only minor issue is the small display doesnt work but who cares its working. Here are the 5 secrets I learned.

I had many errors along the way, this was my last error. "Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint" Hope this helps out.

I used this older version: I might try the newer one later: "https://github.com/OpenTalker/SadTalker/tree/cd4c0465ae0b54a6f85af57f5c65fec9fe23e7f8"

" this talks about SADTALKER_CHECKPOINTS folder

2 "https://huggingface.co/vinthony/SadTalker/tree/main" these files are missing add them to this location ".\SadTalker\SADTALKER_CHECKPOINTS"

3 shout out to sakura1217 for this piece of gold image. follow this too.

4 Redundant files are keys, I am sure there is a method to the madness but better you have extra files and have it working. Plus don't forget to add a ".\SadTalker\results" folder important for your output or it wont work.

5 If you arent sure and you are still having issues here are my Package/version. I am on Win 11 running Cuda

Package Version

absl-py 2.1.0 addict 2.4.0 aiofiles 23.2.1 aiosignal 1.3.1 altair 5.4.1 annotated-types 0.7.0 anyio 4.4.0 attrs 24.2.0 audioread 3.0.1 av 12.3.0 babel 2.16.0 basicsr 1.4.2 better-profanity 0.7.0 bibtexparser 2.0.0b7 blinker 1.8.2 cachetools 5.5.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.3.2 click 8.1.7 clip 1.0 clldutils 3.22.2 cn2an 0.5.22 colorama 0.4.6 coloredlogs 15.0.1 colorlog 6.8.2 contourpy 1.1.1 cycler 0.12.1 decorator 5.1.1 Deprecated 1.2.14 detoxify 0.5.2 dlinfo 1.2.1 einops 0.8.0 eng_to_ipa 0.0.2 exceptiongroup 1.2.2 face-alignment 1.3.5 facexlib 0.3.0 fastapi 0.114.2 ffmpeg-python 0.2.0 ffmpy 0.4.0 filelock 3.16.0 filterpy 1.4.5 flatbuffers 24.3.25 fonttools 4.53.1 frozenlist 1.4.1 fsspec 2024.9.0 ftfy 6.2.3 future 1.0.0 gfpgan 1.3.8 gitdb 4.0.11 GitPython 3.1.43 google-auth 2.34.0 google-auth-oauthlib 1.0.0 gradio 3.41.2 gradio_client 0.5.0 grpcio 1.66.1 h11 0.14.0 httpcore 1.0.5 httpx 0.27.2 huggingface-hub 0.24.7 humanfriendly 10.0 idna 3.10 imageio 2.19.3 imageio-ffmpeg 0.4.7 importlib_metadata 8.5.0 importlib_resources 6.4.5 isodate 0.6.1 jaconv 0.4.0 Jinja2 3.1.4 joblib 1.1.0 jsonschema 4.23.0 jsonschema-specifications 2023.12.1 kiwisolver 1.4.7 kornia 0.6.8 kornia_rs 0.1.5 langid 1.1.6 language-tags 1.2.0 lazy_loader 0.4 librosa 0.9.2 linkify-it-py 2.0.3 llvmlite 0.41.1 lmdb 1.5.1 Markdown 3.7 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.7.5 mdurl 0.1.2 more-itertools 10.5.0 msgpack 1.0.8 multidict 6.0.5 music-tag 0.4.3 mutagen 1.47.0 narwhals 1.8.1 networkx 3.1 numba 0.58.1 numpy 1.23.4 oauthlib 3.2.2 openai-whisper 20231117 opencv-python 4.10.0.84 orjson 3.10.7 packaging 24.1 pandas 2.0.3 phonemizer 3.3.0 pillow 10.4.0 pip 23.0.1 pkgutil_resolve_name 1.3.10 platformdirs 4.3.3 pooch 1.8.2 proces 0.1.7 progressbar 2.5 protobuf 5.28.1 pyarrow 17.0.0 pyasn1 0.6.1 pyasn1_modules 0.4.1 pycparser 2.22 pydantic 2.9.1 pydantic_core 2.23.3 pydeck 0.9.1 pydub 0.25.1 Pygments 2.18.0 pylatexenc 2.10 pyparsing 3.1.4 pyreadline3 3.4.1 python-dateutil 2.9.0.post0 python-multipart 0.0.9 pytz 2024.2 PyWavelets 1.4.1 PyYAML 6.0.2 rdflib 7.0.0 referencing 0.35.1 regex 2024.9.11 requests 2.32.3 requests-oauthlib 2.0.0 resampy 0.3.1 rich 13.8.1 rpds-py 0.10.0 rsa 4.9 ruff 0.6.5 safetensors 0.4.5 scikit-image 0.19.3 scikit-learn 1.1.3 scipy 1.10.1 segments 2.2.1 semantic-version 2.10.0 sentencepiece 0.2.0 setuptools 56.0.0 shellingham 1.5.4 six 1.16.0 smmap 5.0.1 sniffio 1.3.1 soundfile 0.12.1 spandrel 0.3.4 starlette 0.38.5 streamlit 1.38.0 tabulate 0.9.0 taming-transformers 0.0.1 tb-nightly 2.14.0a20230808 tenacity 8.5.0 tensorboard-data-server 0.7.2 threadpoolctl 3.5.0 tifffile 2023.7.10 tiktoken 0.7.0 tokenizers 0.19.1 toml 0.10.2 tomli 2.0.1 tomlkit 0.12.0 torch 1.12.1+cu113 torchaudio 0.12.1+cu113 torchlibrosa 0.1.0 torchsde 0.2.6 torchvision 0.13.1+cu113 tornado 6.4.1 tqdm 4.66.5 trampoline 0.1.2 transformers 4.44.2 typer 0.12.5 typing_extensions 4.12.2 tzdata 2024.1 uc-micro-py 1.0.3 uritemplate 4.1.1 urllib3 2.2.3 uvicorn 0.30.6 voicefixer 0.1.3 watchdog 4.0.2 wavmark 0.0.3 wcwidth 0.2.13 websockets 11.0.3 Werkzeug 3.0.4 wheel 0.44.0 yacs 0.1.8 yapf 0.40.2 zipp 3.20.2

[notice] A new release of pip is available: 23.0.1 -> 24.2 [notice] To update, run: python.exe -m pip install --upgrade pip

OpenTalker / SadTalker

I keep getting this error does anyone have a solution? _ raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint #441

2 "https://huggingface.co/vinthony/SadTalker/tree/main" these files are missing add them to this location ".\SadTalker\SADTALKER_CHECKPOINTS"

3 shout out to sakura1217 for this piece of gold image. follow this too.

4 Redundant files are keys, I am sure there is a method to the madness but better you have extra files and have it working. Plus don't forget to add a ".\SadTalker\results" folder important for your output or it wont work.

5 If you arent sure and you are still having issues here are my Package/version. I am on Win 11 running Cuda