[Bug]: The SadTalker extension hasn't been working for months.

MisterT96 commented 5 months ago

Checklist

[ ] The issue exists after disabling all extensions
[X] The issue exists on a clean installation of webui
[ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
[X] The issue exists in the current version of the webui
[ ] The issue has not been reported before recently
[X] The issue has been reported before but has not been fixed yet

What happened?

SadTalker last worked for me under version 1.7.0 and hasn't worked since. Has anyone found a solution for this or does anyone still have version 1.7.0 and could make it available for download?

I used a new installation of stable-diffusion-webui and encounter errors as soon as I install SadTalker. I am using "Installation on Windows 10/11 with NVidia-GPUs using release package" and Python 3.10.6.

Errors like: AttributeError: module 'numpy' has no attribute 'complex'. np.complex was a deprecated alias for the builtin complex. To avoid this error in existing code, use complex by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.complex128 here. The alias was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

The error seems to occur because SadTalker hasn't been updated for many months.

Who can help?

Steps to reproduce the problem

clean installation Try to install SadTalker with webui.

What should have happened?

SadTalker should have worked.

What browsers do you use to access the UI ?

No response

Sysinfo

All requirements are met.

Console logs

All requirements are met.

Additional information

No response

w-e-w commented 5 months ago

all versions are available in the Repository, no need to ask any one for old version https://github.com/AUTOMATIC1111/stable-diffusion-webui/releases

or you could just switch to the version you want by running a quick command git checkout v1.8.0 in webui root

that being said

AttributeError: module 'numpy' has no attribute 'complex'.
np.complex was a deprecated alias for the builtin complex.

I highly doubt that is the cause of the issue of it not working because numpy version used by webui was last change in the 1.8.0 release, which you said to be working

MisterT96 commented 5 months ago

git checkout v1.8.0

Thank you for that switching method. Regarding the second problem, my error log is:

*** Error executing callback ui_tabs_callback for C:\Users\Username\Desktop\stable-diffusion-webui\extensions\SadTalker\scripts\extension.py
    Traceback (most recent call last):
      File "C:\Users\Username\Desktop\stable-diffusion-webui\modules\script_callbacks.py", line 180, in ui_tabs_callback
        res += c.callback() or []
      File "C:\Users\Username\Desktop\stable-diffusion-webui\extensions\SadTalker\scripts\extension.py", line 172, in on_ui_tabs
        from app import sadtalker_demo
      File "C:\Users\Username\Desktop\stable-diffusion-webui/extensions/SadTalker\app.py", line 3, in <module>
        from src.gradio_demo import SadTalker
      File "C:\Users\Username\Desktop\stable-diffusion-webui/extensions/SadTalker\src\gradio_demo.py", line 6, in <module>
        from src.generate_batch import get_data
      File "C:\Users\Username\Desktop\stable-diffusion-webui/extensions/SadTalker\src\generate_batch.py", line 8, in <module>
        import src.utils.audio as audio
      File "C:\Users\Username\Desktop\stable-diffusion-webui/extensions/SadTalker\src\utils\audio.py", line 1, in <module>
        import librosa
      File "C:\Users\Username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\librosa\__init__.py", line 211, in <module>
        from . import core
      File "C:\Users\Username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\librosa\core\__init__.py", line 9, in <module>
        from .constantq import *  # pylint: disable=wildcard-import
      File "C:\Users\Username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\librosa\core\constantq.py", line 1058, in <module>
        dtype=np.complex,
      File "C:\Users\Username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\numpy\__init__.py", line 338, in __getattr__
        raise AttributeError(__former_attrs__[attr])
    AttributeError: module 'numpy' has no attribute 'complex'.
    `np.complex` was a deprecated alias for the builtin `complex`. To avoid this error in existing code, use `complex` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.complex128` here.
    The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
        https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

I highly doubt that is the cause of the issue of it not working because numpy version used by webui was last change in the 1.8.0 release, which you said to be working

Yeah, my mistake. Version 1.8.0 is not working. But version 1.7.0 is working all fine. So thank you for writing me that version change command. A+ experience!

w-e-w commented 5 months ago

       File "C:\Users\Username\Desktop\stable-diffusion-webui/extensions/SadTalker\src\utils\audio.py", line 1, in <module>
        import librosa

seeing this I think the issue is version compatibility issue with librosa==0.9.2 and numpy==1.26.2

enszio commented 3 months ago

In case anyone is still interested in this, I decided to do a little digging into it and was able to work around the problem. As @w-e-w mentioned there is a compatibility issue with librosa==0.9.2 and newer versions of numpy. The same compatibility problem exists in SadTalker as well.

For librosa, I just changed everywhere that used np.float to float and np.complex to complex. You won't need to change np.float32, np.float64, np.complex64, np.complex128, etc. Unfortunately, I wasn't keeping track of which files I changed, but there were only a couple places to change, so I think you will find them quickly. Also, I think you might be able to upgrade librosa to the latest version to fix it, but I didn't try it.

For SadTalker, there was an additional change since newer versions of python do not allow "ragged arrays", but it turned out to be a simple change. The diff for both changed I made in SadTalkerfollows:

diff --git a/src/face3d/util/my_awing_arch.py b/src/face3d/util/my_awing_arch.py
index cd56561..647fa11 100644
--- a/src/face3d/util/my_awing_arch.py
+++ b/src/face3d/util/my_awing_arch.py
@@ -15,7 +15,7 @@ def calculate_points(heatmaps):
     indexes = np.argmax(heatline, axis=2)

     preds = np.stack((indexes % W, indexes // W), axis=2)
-    preds = preds.astype(np.float, copy=False)
+    preds = preds.astype(float, copy=False)

     inr = indexes.ravel()

diff --git a/src/face3d/util/preprocess.py b/src/face3d/util/preprocess.py
index b77a3a4..29fa110 100644
--- a/src/face3d/util/preprocess.py
+++ b/src/face3d/util/preprocess.py
@@ -98,6 +98,6 @@ def align_img(img, lm, lm3D, mask=None, target_size=224., rescale_factor=102.):

     # processing the image
     img_new, lm_new, mask_new = resize_n_crop_img(img, lm, t, s, target_size=target_size, mask=mask)
-    trans_params = np.array([w0, h0, s, t[0], t[1]])
+    trans_params = np.array([w0, h0, s, float(t[0]), float(t[1])])

     return trans_params, img_new, lm_new, mask_new

Anyway, good luck out there

w-e-w commented 3 months ago

tip markdown diff code block can be highlighted

```diff
     no change
+    add stuff
-    remove stuff

```diff
     no change
+    add stuff
-    remove stuff

enszio commented 3 months ago

@w-e-w Thanks for the tip!

AUTOMATIC1111 / stable-diffusion-webui