search

Bujiazi / MotionClone

Official implementation of MotionClone: Training-Free Motion Cloning for Controllable Video Generation
290 stars 14 forks source link

Missing code files #3

Closed piyushK52 closed 2 weeks ago

piyushK52 commented 3 weeks ago

I am trying to run the provided code, but a couple of files seem to be missing. In invert.py and pipeline.py you guys are importing UNet3DConditionModel and SparseControlNetModel from motionclone.models, but motionclone/models directory doesn't exist. Please cross-check the code to fix this issue or update the readme (in case there is a missing step) . Screenshot 2024-06-30 at 1 00 07 AM Screenshot 2024-06-30 at 1 00 23 AM

Bujiazi commented 3 weeks ago

It appears that there was an issue with our .gitignore file, which has now been resolved 😂. Thank you for your feedback.

piyushK52 commented 3 weeks ago

Thanks for the response. Yeah it seems to be fixed. Can you give an estimate on how much vram it requires? I am running the fox example but even on a rtx 4090 (24gb vram) it is going out of memory

Bujiazi commented 2 weeks ago

Thanks for your feedback. When generating videos configured at 16 512 512, the peak GPU memory usage of MotionClone is approximately 30+ GB. You might consider reducing the video resolution to run it on your GPU. (We have updated motionclone/pipelines/additional_components.py to support lower resolutions like 320 * 320)

piyushK52 commented 2 weeks ago

That is very helpful, thanks. Btw I was able to make the generations work on A100 (80 GB). Closing this issue.