search

Tangshitao / MVDiffusion

MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion, NeurIPS 2023 (spotlight)
447 stars 21 forks source link

Train MVDiffusion w/ random camera trajectory but w/o depth cond #45

Open OrangeSodahub opened 3 months ago

OrangeSodahub commented 3 months ago

Hi, I'm very interested in your work. And I want to know that if I could train your depth version MVDiffusion model but using SD without depth cond. (e.g. SDv1.5)? And if yes, do you have any advice on how to sample images from the entire camera trajectory to perform consistency well according to your experience. And the last question is, if the precision of depth values matter in training process? Maybe I use some depth values from bounding box to train.

Tangshitao commented 3 months ago

I don't think current pipeline can be used to train images without depths.

OrangeSodahub commented 3 months ago

@Tangshitao Sorry I don't exactly understand what you mean. Let me clarify something, I mean using depths to calculate the correspondence but don't use depths as the condition when generating images. Do you mean without depth cond. the results are not expected as with depth cond.?