MyNiuuu / MOFA-Video

Official Pytorch implementation for MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model.
https://myniuuu.github.io/MOFA_Video
Other
358 stars 22 forks source link

unable to run due to errors #10

Closed sushanthpy closed 1 week ago

sushanthpy commented 1 week ago

Great Work ! :). I liked the concept and started experimenting, but I'm encountering the following issue. I need your help to fix it.

xhzgas xyz233 Screenshot 2024-06-17 2156133444

without paint mark

File "/mnt/d/MOFA-Video/MOFA-Video-Traj/venv/lib/python3.10/site-packages/scipy/interpolate/cubic.py", line 249, in init x, , y, axis, _ = prepare_input(x, y, axis) File "/mnt/d/MOFA-Video/MOFA-Video-Traj/venv/lib/python3.10/site-packages/scipy/interpolate/_cubic.py", line 55, in prepare_input raise ValueError("x must contain at least 2 elements.") ValueError: x must contain at least 2 elements.

with paint mark es/gradio/utils.py", line 832, in wrapper response = f(*args, **kwargs) File "/mnt/d/MOFA-Video/MOFA-Video-Traj/run_gradio.py", line 507, in run divide_points_afterinterpolate(resized_all_points_384, motion_brush_mask_384) File "/mnt/d/MOFA-Video/MOFA-Video-Traj/run_gradio.py", line 43, in divide_points_afterinterpolate starts = resized_all_points[:, 0] # [K, 2] IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed

MyNiuuu commented 1 week ago

this image From this image it appears that you haven't correctly added the trajectories to the image. I'm curious about how you managed to create these blue lines instead of trajectories or masks in the images. The correct interface looks like this after correctly adding the trajectories to the image (red arrows are the visualization of the trajectories):

Screen Shot 2024-06-18 at 14 51 22

Additionally, which version of Gradio are you using? gradio==4.5.0 is recommended for this demo.

sushanthpy commented 1 week ago

Hi @MyNiuuu Thank you for the response. i have upgraded to gradio==4.36.1 based on the warning, let me roll it back and check.

I have manually added the trajectories using pain tool. let me follow instructions and retry and keep you posted.

sushanthpy commented 1 week ago

Followed the instructions and downgraded to gradio 4.5 , when i click on trajectories getting below error message

base2 basenew downgrade

MyNiuuu commented 1 week ago

You have to first press the 'Add Trajectory button' before clicking the image to add trajectories. I tried the demo on my computer and encountered the same error when I clicked the image directly without first pressing the 'Add Trajectory' button. Also, to prevent possible errors, you can follow the demo's detailed instructions step by step:

Screen Shot 2024-06-18 at 14 51 22
sushanthpy commented 1 week ago

Thank you for the prompt response and support, everything working has expected :) final output