Pretrained model produces noises

TencentARC / MotionCtrl

MotionCtrl: A Unified and Flexible Motion Controller for Video Generation

https://wzhouxiff.github.io/projects/MotionCtrl/

Apache License 2.0

1.15k stars 66 forks source link

Pretrained model produces noises #13

Closed SnowdenLee closed 5 months ago

SnowdenLee commented 5 months ago

Hi @wzhouxiff , thanks for the interesting work! I downloaded the model here https://huggingface.co/TencentARC/MotionCtrl/blob/main/motionctrl_svd.ckpt for SVD+MotionCtrl inference. All keys add up, but the results are basically noises. Do you maybe have an idea what causes this? Thanks a lot!

Netanel-Tamir commented 5 months ago

Hi! I've encountered the same issue. How did you solve it?

SnowdenLee commented 4 months ago

Hi @Netanel-Tamir , I managed to fix the issue by using torch 2.0.1. Seems like a CUDA compatibility problem.

Netanel-Tamir commented 4 months ago

Thanks @SnowdenLee, can you send the list of packages and versions that worked for you?

SnowdenLee commented 4 months ago

Hi @Netanel-Tamir Here is the full list requirement.txt, there are some redundant packages that are not used by this repo though.

Netanel-Tamir commented 4 months ago

Thanks for the help!

flysssss commented 2 months ago

@SnowdenLee I've encountered the same issue. Can you tell me your successful CUDA version？I tried to upgrade torch 2.0.1 with the version you provided, and a CUDA kernel error occurred while generating reasoning ![Uploading image.png…]()

flysssss commented 2 months ago

wzhouxiff commented 2 months ago

Our environment is following the requirements provided by SVD: https://github.com/Stability-AI/generative-models/blob/0.0.1/requirements_pt13.txt https://github.com/Stability-AI/generative-models/blob/0.0.1/requirements_pt2.txt