search

MCG-NJU / BIVDiff

[CVPR 2024] BIVDiff: A Training-free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models
Apache License 2.0
51 stars 1 forks source link

outpainting video is very blurry #1

Closed ioir123ju closed 2 months ago

ioir123ju commented 3 months ago

source source_video idm video_by_idm mask masked_video bivdiff video_by_bivdiff

ShiFengyuan1999 commented 3 months ago

Hi @ioir123ju, The blurry results may come from poor frames by IDM. There are big differences among the areas completed by IDM in each frame. To make these particularly different images temporally consistent, VDM may tend to generate blurry videos.

Video generated by IDM idm

ioir123ju commented 3 months ago

Hi @ioir123ju, The blurry results may come from poor frames by IDM. There are big differences among the areas completed by IDM in each frame. To make these particularly different images temporally consistent, VDM may tend to generate blurry videos.

Video generated by IDM idm

If I generate a decent outpainting images in ComfyUI, can I directly use VDM for smoothing?Or I can also share my workflow with you; it can generate excellent outpainting images. image

ShiFengyuan1999 commented 3 months ago

@ioir123ju If there are no big differences (in structures, objects, and so on) among the filled areas of the outpainting images by ComfyUI, you can use VDM for smoothing and get better results.

ioir123ju commented 3 months ago

@ioir123ju If there are no big differences (in structures, objects, and so on) among the filled areas of the outpainting images by ComfyUI, you can use VDM for smoothing and get better results.

I'm not sure where I went wrong; the generated image is just noise. image

ShiFengyuan1999 commented 3 months ago

@ioir123ju Did you remove the codes of Mixed Inversion. Once you get the videos by IDM, you should first invert the video to noise latents by Mixed Inversion, then use VDM for temporal smoothing.