ComfyUI-MuseTalk_FSH

the comfyui custom node of MuseTalk to make audio driven videos!

How to use

make sure ffmpeg is worked in your commandline for Linux

apt update
apt install ffmpeg

for Windows,you can install ffmpeg by WingetUI automatically

then!

git clone https://github.com/AIFSH/ComfyUI-MuseTalk_FSH.git
cd ComfyUI-MuseTalk_FSH
pip install -r requirements.txt

mmlab packages

pip install --no-cache-dir -U openmim 
mim install mmengine 
mim install "mmcv>=2.0.1" 
mim install "mmdet>=3.1.0" 
mim install "mmpose>=1.1.0" 

Download weights

You can download weights manually as follows:

1. Download our trained weights.

2. Download the weights of other components:

   • sd-vae-ft-mse
   • whisper
   • dwpose
   • face-parse-bisent
   • resnet18

或者下载MuseTalk.zip， 解压后把子文件夹放入ComfyUI-MuseTalk_FSH/models/目录

Finally, these weights should be organized in models as follows:

ComfyUI-MuseTalk_FSH/models/
├── musetalk
│   └── musetalk.json
│   └── pytorch_model.bin
├── dwpose
│   └── dw-ll_ucoco_384.pth
├── face-parse-bisent
│   ├── 79999_iter.pth
│   └── resnet18-5c106cde.pth
├── sd-vae-ft-mse
│   ├── config.json
│   └── diffusion_pytorch_model.bin
└── whisper
    └── tiny.pt

Tutorial

• Demo on 3060 12GB
• Demo on 4090 24GB

WeChat Group && Donate

Thanks

• MuseTalk