search

RuoyuFeng / CCEdit

CCEdit: Creative and Controllable Video Editing via Diffusion Models
Other
76 stars 2 forks source link

What is the reason when I set num_keyframes=1, and output is just noise? #2

Open justinday123 opened 1 month ago

justinday123 commented 1 month ago

Can I know the reason?

RuoyuFeng commented 1 month ago

Can I know the reason?

@justinday123 hi, thanks for your question. The parameter of num_keyfames indicates the number of frames in the sampling process. And during training, the input data contains 17 frames in our setting. So if you set the num_keyframes as 1, it would cause training-inference mismatch, then the results tends to be noise. Hope can answer your question.

justinday123 commented 1 month ago

Thank you for your answer. Will ablation study for number of keyframes will be updated? Also, the number of the frames of the result at inference time will be the size of keyframes? Thank you.