search

Tangshitao / MVDiffusion

MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion, NeurIPS 2023 (spotlight)
447 stars 21 forks source link

generate missing key error using pano_outpaint.ckpt #16

Closed pengwangucla closed 10 months ago

pengwangucla commented 10 months ago

https://github.com/Tangshitao/MVDiffusion/blob/1a535e48750d2f0485bccfb70b19333b46106699/demo.py#L78C40-L78C40

Should it be 'False' if vae is not trained ? Otherwise, there is a key mismatch issue. 

        Missing key(s) in state_dict: "vae.encoder.mid_block.attentions.0.to_q.weight", "vae.encoder.mid_block.attentions.0.to_q.bias", "vae.encoder.mid_block.attentions.0.to_k.weight", 
"vae.encoder.mid_block.attentions.0.to_k.bias", "vae.encoder.mid_block.attentions.0.to_v.weight", "vae.encoder.mid_block.attentions.0.to_v.bias", "vae.encoder.mid_block.attentions.0.to_out.0.weight", 
"vae.encoder.mid_block.attentions.0.to_out.0.bias", "vae.decoder.mid_block.attentions.0.to_q.weight", "vae.decoder.mid_block.attentions.0.to_q.bias", "vae.decoder.mid_block.attentions.0.to_k.weight", 
"vae.decoder.mid_block.attentions.0.to_k.bias", "vae.decoder.mid_block.attentions.0.to_v.weight", "vae.decoder.mid_block.attentions.0.to_v.bias", "vae.decoder.mid_block.attentions.0.to_out.0.weight", 
"vae.decoder.mid_block.attentions.0.to_out.0.bias". 
        Unexpected key(s) in state_dict: "vae.encoder.mid_block.attentions.0.query.weight", "vae.encoder.mid_block.attentions.0.query.bias", "vae.encoder.mid_block.attentions.0.key.weight", 
"vae.encoder.mid_block.attentions.0.key.bias", "vae.encoder.mid_block.attentions.0.value.weight", "vae.encoder.mid_block.attentions.0.value.bias", "vae.encoder.mid_block.attentions.0.proj_attn.weight", 
"vae.encoder.mid_block.attentions.0.proj_attn.bias", "vae.decoder.mid_block.attentions.0.query.weight", "vae.decoder.mid_block.attentions.0.query.bias", "vae.decoder.mid_block.attentions.0.key.weight", 
"vae.decoder.mid_block.attentions.0.key.bias", "vae.decoder.mid_block.attentions.0.value.weight", "vae.decoder.mid_block.attentions.0.value.bias", "vae.decoder.mid_block.attentions.0.proj_attn.weight", 
"vae.decoder.mid_block.attentions.0.proj_attn.bias".
Tangshitao commented 10 months ago

Yes, please see https://github.com/Tangshitao/MVDiffusion/issues/12