OpenTalker / SadTalker

[CVPR 2023] SadTalker:Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation
https://sadtalker.github.io/
Other
12.01k stars 2.24k forks source link

For those closed Issues by mistake. #152

Open vinthony opened 1 year ago

vinthony commented 1 year ago

Hi everyone,

Sorry for the installation of the sd-webui problem and other platforms (eg, mac), we are working hard to make every user enjoy the power of sadtalker.

So if there are any issues unsolved by mistake closed or there are unsolved comments in the closed issues, please feel free to list them here.

We will try to solve it.

wukenny11111 commented 1 year ago

D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/auido2pose_00140-model.pth D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/shape_predictor_68_face_landmarks.dat D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/facevid2vid_00189-model.pth.tar C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpvvqcrjzd.png

Traceback (most recent call last): File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\routes.py", line 337, in run_predict output = await app.get_blocks().process_api( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\blocks.py", line 1015, in process_api result = await self.call_function( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\blocks.py", line 833, in call_function prediction = await anyio.to_thread.run_sync( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, args) File "D:\AI\novelai-webui-aki-v3\modules\call_queue.py", line 15, in f res = func(args, **kwargs) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\gradio_demo.py", line 111, in test first_coeff_path, crop_pic_path, crop_info = self.preprocess_model.generate(pic_path, first_frame_dir, preprocess) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\utils\preprocess.py", line 121, in generate lm = self.kp_extractor.extract_keypoint(frames_pil, landmarks_path) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\face3d\extract_kp_videos_safe.py", line 46, in extract_keypoint current_kp = self.extract_keypoint(image) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\face3d\extract_kp_videos_safe.py", line 64, in extract_keypoint bboxes = bboxes[0] IndexError: index 0 is out of bounds for axis 0 with size 0 提示:Python 运行时抛出了一个异常。请检查疑难解答页面。

在SD中运行报错

vinthony commented 1 year ago

D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/auido2pose_00140-model.pth D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/shape_predictor_68_face_landmarks.dat D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/facevid2vid_00189-model.pth.tar C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpvvqcrjzd.png

Traceback (most recent call last): File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\routes.py", line 337, in run_predict output = await app.get_blocks().process_api( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\blocks.py", line 1015, in process_api result = await self.call_function( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\blocks.py", line 833, in call_function prediction = await anyio.to_thread.run_sync( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, args) File "D:\AI\novelai-webui-aki-v3\modules\call_queue.py", line 15, in f res = func(args, **kwargs) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\gradio_demo.py", line 111, in test first_coeff_path, crop_pic_path, crop_info = self.preprocess_model.generate(pic_path, first_frame_dir, preprocess) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\utils\preprocess.py", line 121, in generate lm = self.kp_extractor.extract_keypoint(frames_pil, landmarks_path) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\face3d\extract_kp_videos_safe.py", line 46, in extract_keypoint current_kp = self.extract_keypoint(image) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\face3d\extract_kp_videos_safe.py", line 64, in extract_keypoint bboxes = bboxes[0] IndexError: index 0 is out of bounds for axis 0 with size 0 提示:Python 运行时抛出了一个异常。请检查疑难解答页面。

在SD中运行报错

这个是因为检测不到人脸,请确认输入的图像是”真人“或者类真人图像。

wukenny11111 commented 1 year ago

D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/auido2pose_00140-model.pth D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/shape_predictor_68_face_landmarks.dat D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/facevid2vid_00189-model.pth.tar C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpvvqcrjzd.png Traceback (most recent call last): File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\routes.py", line 337, in run_predict output = await app.get_blocks().process_api( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\blocks.py", line 1015, in process_api result = await self.call_function( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\blocks.py", line 833, in call_function prediction = await anyio.to_thread.run_sync( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, args) File "D:\AI\novelai-webui-aki-v3\modules\call_queue.py", line 15, in f res = func(args, **kwargs) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\gradio_demo.py", line 111, in test first_coeff_path, crop_pic_path, crop_info = self.preprocess_model.generate(pic_path, first_frame_dir, preprocess) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\utils\preprocess.py", line 121, in generate lm = self.kp_extractor.extract_keypoint(frames_pil, landmarks_path) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\face3d\extract_kp_videos_safe.py", line 46, in extract_keypoint current_kp = self.extract_keypoint(image) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\face3d\extract_kp_videos_safe.py", line 64, in extract_keypoint bboxes = bboxes[0] IndexError: index 0 is out of bounds for axis 0 with size 0 提示:Python 运行时抛出了一个异常。请检查疑难解答页面。 在SD中运行报错

这个是因为检测不到人脸,请确认输入的图像是”真人“或者类真人图像。

照片是用SD生成的写实照片,分辨率是20482048,我把它改成512512 还是报错!

vinthony commented 1 year ago

有图片吗?

2catycm commented 10 months ago

D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/auido2pose_00140-model.pth D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/shape_predictor_68_face_landmarks.dat D:\AI\novelai-webui-aki-v3/extensions/SadTalker/checkpoints/facevid2vid_00189-model.pth.tar C:\Users\ADMINI~1\AppData\Local\Temp\2\tmpvvqcrjzd.png Traceback (most recent call last): File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\routes.py", line 337, in run_predict output = await app.get_blocks().process_api( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\blocks.py", line 1015, in process_api result = await self.call_function( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\gradio\blocks.py", line 833, in call_function prediction = await anyio.to_thread.run_sync( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "D:\AI\novelai-webui-aki-v3\py310\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, args) File "D:\AI\novelai-webui-aki-v3\modules\call_queue.py", line 15, in f res = func(args, **kwargs) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\gradio_demo.py", line 111, in test first_coeff_path, crop_pic_path, crop_info = self.preprocess_model.generate(pic_path, first_frame_dir, preprocess) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\utils\preprocess.py", line 121, in generate lm = self.kp_extractor.extract_keypoint(frames_pil, landmarks_path) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\face3d\extract_kp_videos_safe.py", line 46, in extract_keypoint current_kp = self.extract_keypoint(image) File "D:\AI\novelai-webui-aki-v3/extensions/SadTalker\src\face3d\extract_kp_videos_safe.py", line 64, in extract_keypoint bboxes = bboxes[0] IndexError: index 0 is out of bounds for axis 0 with size 0 提示:Python 运行时抛出了一个异常。请检查疑难解答页面。 在SD中运行报错

这个是因为检测不到人脸,请确认输入的图像是”真人“或者类真人图像。

作者您好,请问如果要修改SadTalker,让它支持非真人的动漫图像,应该从哪些模块下手呢

cstuart1310 commented 7 months ago

这个是因为检测不到人脸,请确认输入的图像是”真人“或者类真人图像。 English Translation: "This is because human faces cannot be detected. Please confirm that the input image is a "real person" or a human-like image."

Thank you, I added 1 second of a still frame of the person's face to the beginning of the reference video, and it has been able to detect the face and carry on with the animations in the rest of the video :)