search

G-U-N / Motion-I2V

[SIGGRAPH 2024] Motion I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling
https://xiaoyushi97.github.io/Motion-I2V/
117 stars 11 forks source link

Instructions on the project #1

Open xizaoqu opened 4 months ago

xizaoqu commented 4 months ago

Hi, thanks for sharing the great work. Could you provide more instructions on the project? (i.e. how to do inference, how to download checkpoints)

G-U-N commented 4 months ago

Thanks for the interest! I will write instructions asap.

xizaoqu commented 4 months ago

Thanks for the interest! I will write instructions asap.

Hi, could you help me to check the weird results?

image image

I suppose the weights are loaded successfully? image

TomSuen commented 4 months ago

Hi, could you tell me how to run the gradio？How much memory will it use? @xizaoqu

TomSuen commented 4 months ago

Hi, could you help me to check the weird results?

I have this error 1721290074952_85530E35-5A3E-4c6b-A43E-DB9428B057CC

how to fix it?

xizaoqu commented 4 months ago

how to fix it?

Hi, try setting the CUDA_HOME to your cuda dir. I do not produce the right results so I'm not sure if I do it right. I just run app.py.

TomSuen commented 4 months ago

Hi, try setting the CUDA_HOME to your cuda dir. I do not produce the right results so I'm not sure if I do it right. I just run app.py.

Thank you, for your drag result, I think you can add a brush mask on the face. It may work well. 522dcaffb23323425933ad1d643bda7 image

xizaoqu commented 4 months ago

Thank you, for your drag result, I think you can add a brush mask on the face. It may work well.

In my case, adding mask can't fix the problem.

image

image (2)

woman2

TomSuen commented 4 months ago

In my case, adding mask can't fix the problem.

Maybe you should remove the prompt and adjust the Flow unit, it seems to affect range of motion.

Fancy93 commented 4 months ago

In my case, adding mask can't fix the problem.

Maybe you should remove the prompt and adjust the Flow unit, it seems to affect range of motion.

Hi, I would like to ask if you have encountered the following error? image image

TomSuen commented 4 months ago

Hi, I would like to ask if you have encountered the following error?

Sorry, i haven't meet with this error yet. It seems that something incompatible between CUDA and pytorch, if you use linux, you can run which nvcc to check the CUDA path, and run nvcc --version to check current CUDA version.

I use docker, and my nvcc --version is above 11.8, pytorch==1.13.1+cu117