不支持加载SVD的xxx.fp16.safetensors？

[jkla139]

运行提示SVD/stable-video-diffusion-img2vid-xt-1-1下找不到???.bin文件，但是 https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1 只有safetensors文件啊

[zyayoung]

能否提供您遇到的具体的错误信息和traceback？

[jkla139]

能否提供您遇到的具体的错误信息和traceback？

我只下载了 fp16的safetensors，似乎 XXX.from_pretrained 不能查找半精度模型……

An error occurred while trying to fetch models/SVD/stable-video-diffusion-img2vid-xt-1-1: Error no file named diffusion_pytorch_model.safetensors found in directory models/SVD/stable-video-diffusion-img2vid-xt-1-1.
Defaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead.
Traceback (most recent call last):
  File "/root/MimicMotion/inference.py", line 134, in <module>
    main(args)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/MimicMotion/inference.py", line 89, in main
    pipeline = create_pipeline(infer_config, device)
  File "/root/MimicMotion/mimicmotion/utils/loader.py", line 42, in create_pipeline
    mimicmotion_models = MimicMotionModel(infer_config.base_model_path).to(device=device).eval()
  File "/root/MimicMotion/mimicmotion/utils/loader.py", line 24, in __init__
    self.vae = AutoencoderKLTemporalDecoder.from_pretrained(
  File "/root/miniconda3/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/diffusers/models/modeling_utils.py", line 722, in from_pretrained
    model_file = _get_model_file(
  File "/root/miniconda3/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/diffusers/utils/hub_utils.py", line 310, in _get_model_file
    raise EnvironmentError(
OSError: Error no file named diffusion_pytorch_model.bin found in directory models/SVD/stable-video-diffusion-img2vid-xt-1-1.

[jkla139]

测试发现只有 unet 里面的 xxx.fp16.safetensors 文件是可以查找到的，不清楚其它模型为什么不支持查找半精度模型？ @huggingface 把 SVD 里的其它几个模型换成 fp32的就可以运行了

[ricardo-ml]

mimicmotion/utils/loader.py文件改成下面这样： self.vae = AutoencoderKLTemporalDecoder.from_pretrained(base_model_path, subfolder="vae", variant="fp16") self.image_encoder = CLIPVisionModelWithProjection.from_pretrained(base_model_path, subfolder="image_encoder",variant="fp16") self.noise_scheduler = EulerDiscreteScheduler.from_pretrained(base_model_path, subfolder="scheduler",variant="fp16") self.feature_extractor = CLIPImageProcessor.from_pretrained(base_model_path, subfolder="feature_extractor",variant="fp16")

[jkla139]

mimicmotion/utils/loader.py文件改成下面这样： self.vae = AutoencoderKLTemporalDecoder.from_pretrained(base_model_path, subfolder="vae", variant="fp16") self.image_encoder = CLIPVisionModelWithProjection.from_pretrained(base_model_path, subfolder="image_encoder",variant="fp16") self.noise_scheduler = EulerDiscreteScheduler.from_pretrained(base_model_path, subfolder="scheduler",variant="fp16") self.feature_extractor = CLIPImageProcessor.from_pretrained(base_model_path, subfolder="feature_extractor",variant="fp16")

Thank you very much ! It works.