search

OpenTalker / SadTalker

[CVPR 2023] SadTalker：Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation
https://sadtalker.github.io/
Other
11.12k stars 2.08k forks source link

How to force to use directml in SadTalker instead of CUDA or CPU? #918

Open cjimenezcmyk opened 1 month ago

cjimenezcmyk commented 1 month ago

Hello, is there any way to use Directml with SadTalker?

Currently I am using the cpu version of Torch, but it is extremely slow. (30min for a 10s clip)

I have an AMD GPU (without budget to buy a Nvidia GPU) , so in the meantime I would like to use directml to speed up a little bit the rendering.

I installed torch-directml, but probably I would need to perform other changes in the code because SadTalker still uses the CPU.

Any advise will be appreciated.

Thanks!

eoffermann commented 1 month ago

"Hello, is there any way to use Directml with SadTalker?" As with most things software-related, the answer to this technically is "yes". I asked ChatGPT to give me an outline of what you'd need to do with this project: https://chatgpt.com/share/2da065ef-4489-4df7-865d-5022dcfd3ad2

That's just a rough overview, but it does a good job of outlining what you'd need to do. You'll have to go through the code implementing DirectML support throughout. It's not a HUGE task but you might find that it's more than you want to deal with. Any source file that uses pytorch will almost certainly need to be touched. Anywhere that it specifies what card to use.

In my install I have torch==2.0.1+cu118, torchaudio==2.0.2+cu118, torchvision==0.15.2+cu118 - but that environment supports multiple ML apps that I've unified to run in a common codebase. I think the instructions require torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1

Either way, you'll need to find a set of dependencies that work if you use the directml versions. torch-directml==0.2.0 and so on. That could be most of the real drudgery - figuring out which torch directml libraries work with what other libraries and the current source.

cjimenezcmyk commented 4 weeks ago

Thanks for the comment eoffermann, I was expecting some sort of argument that could be used in the bat file. (similar as with Stable Diffusion).

I am just a curious pc user adventuring and playing with the new AI tools, with almost zero skills in python or programming in general, so performing the suggested changes by myself will be almost impossible.

But, with your feedback I hope someone skilled could contribute to the project to implement a directml version of sadtalker in the near future.