levihsu / OOTDiffusion

Official implementation of OOTDiffusion: Outfitting Fusion based Latent Diffusion for Controllable Virtual Try-on
Other
5.53k stars 809 forks source link

dc model inference problem #90

Closed frankchieng closed 7 months ago

frankchieng commented 7 months ago

when i run the gradio_ootd.py, it raise error as below: raise ValueError( ValueError: Cannot load ../checkpoints/ootd/ootd_dc/checkpoint-36000 because conv_in.weight expected shape tensor(..., device='meta', size=(320, 4, 3, 3)), but got torch.Size([320, 8, 3, 3]). If you want to instead overwrite randomly initialized weights, please make sure to pass both low_cpu_mem_usage=False and ignore_mismatched_sizes=True. For more information, see also: https://github.com/huggingface/diffusers/issues/1619#issuecomment-1345604389 as an example.

i tried to modify the ootd/inference_ootd_dc.py with

unet_garm = UNetGarm2DConditionModel.from_pretrained(
      UNET_PATH,
      subfolder="unet_garm",
      torch_dtype=torch.float16,
      use_safetensors=True,
      low_cpu_mem_usage=False,
      ignore_mismatched_sizes=True
  )
  unet_vton = UNetVton2DConditionModel.from_pretrained(
      UNET_PATH,
      subfolder="unet_vton",
      torch_dtype=torch.float16,
      use_safetensors=True,
      low_cpu_mem_usage=False,
      ignore_mismatched_sizes=True
  )

then i can run gradio successfully, but the Full-body generation with DC model renders quality is so badly, any clue how i can solve this issue?

levihsu commented 7 months ago

https://github.com/levihsu/OOTDiffusion/issues/82#issuecomment-1975955294 Don't pass low_cpu_mem_usage=False/ignore_mismatched_sizes=True. Please check your files. Each model has different config.json