search

korrawe / guided-motion-diffusion

Other
141 stars 9 forks source link

Generating very long sequences #18

Closed nghorbani closed 1 month ago

nghorbani commented 1 month ago

Hi, Thank you for sharing your work. I am interested to know if gdm could be used to generate very long motion sequences? say 1000 frames; e.g. a character running a trajectory for 1000 frames? I thought of using a sliding window but the speed during the blending section wont be consistent. I would appreciate any help.

nghorbani commented 1 month ago

https://github.com/priorMDM/priorMDM suggests a sequential composition with double take to ensure consistency. Could you imagine a combination of two works?

korrawe commented 1 month ago

Hi @nghorbani, yes, in principle, we should be able to generate very long sequences by out-painting from the existing output like in PriorMDM. Other methods like DiffCollage should also work.

The important thing for GMD specifically is we use absolute motion representation which assumes that we always start at the original facing -Z at the starting frame. So in order to outpaint, we will need to normalize the starting frame of the previous section to be at the origin first. Other than that I believe it will work.

nghorbani commented 1 month ago

does it make sense to make a proof of concept script?

korrawe commented 1 month ago

sure, you can start by checking how the normalization is done in motion_representation.ipynb