RuntimeError: "exp_vml_cpu" not implemented for 'Half'

[ucasiggcas]

hi，dear 大佬，这个是啥问题啊，

>>> video = pipe(
...     input_image=image.resize((512, 512)),
...     num_frames=128, fps=30, height=512, width=512,
...     motion_bucket_id=127,
...     num_inference_steps=50,
...     min_cfg_scale=2, max_cfg_scale=2, contrast_enhance_scale=1.2
... )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/envs/diffsynth/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/data/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 158, in __call__
    image_emb_clip_posi = self.encode_image_with_clip(input_image)
  File "/data/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 53, in encode_image_with_clip
    image = SVDCLIPImageProcessor().resize_with_antialiasing(image, (224, 224))
  File "/dat/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 226, in resize_with_antialiasing
    input = self._gaussian_blur2d(input, ks, sigmas)
  File "/da/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 302, in _gaussian_blur2d
    kernel_x = self._gaussian(kx, sigma[:, 1].view(bs, 1))
  File "/datg/DiffSynth-Studio/diffsynth/pipelines/stable_video_diffusion.py", line 289, in _gaussian
    gauss = torch.exp(-x.pow(2.0) / (2 * sigma.pow(2.0)))

[ucasiggcas]

Package Version

---

altair 5.3.0 attrs 23.2.0 blinker 1.8.2 Brotli 1.0.9 cachetools 5.3.3 certifi 2024.6.2 charset-normalizer 2.0.4 click 8.1.7 controlnet_aux 0.0.7 cupy-cuda12x 13.2.0 diffsynth 1.0.0 einops 0.8.0 fastrlock 0.8.2 filelock 3.13.1 fsspec 2024.6.0 gitdb 4.0.11 GitPython 3.1.43 gmpy2 2.1.2 huggingface-hub 0.23.4 idna 3.7 imageio 2.34.2 imageio-ffmpeg 0.5.1 importlib_metadata 8.0.0 Jinja2 3.1.4 jsonschema 4.22.0 jsonschema-specifications 2023.12.1 lazy_loader 0.4 markdown-it-py 3.0.0 MarkupSafe 2.1.3 mdurl 0.1.2 mkl-fft 1.3.8 mkl-random 1.2.4 mkl-service 2.4.0 mpmath 1.3.0 networkx 3.2.1 numpy 1.26.4 opencv-python 4.10.0.84 packaging 24.1 pandas 2.2.2 pillow 10.3.0 pip 24.0 protobuf 5.27.2 psutil 6.0.0 pyarrow 16.1.0 pydeck 0.9.1 Pygments 2.18.0 PySocks 1.7.1 python-dateutil 2.9.0.post0 pytz 2024.1 PyYAML 6.0.1 referencing 0.35.1 regex 2024.5.15 requests 2.32.2 rich 13.7.1 rpds-py 0.18.1 safetensors 0.4.3 scikit-image 0.24.0 scipy 1.13.1 sentencepiece 0.2.0 setuptools 69.5.1 six 1.16.0 smmap 5.0.1 streamlit 1.36.0 streamlit-drawable-canvas 0.9.3 sympy 1.12 tenacity 8.4.2 tifffile 2024.6.18 timm 1.0.7 tokenizers 0.19.1 toml 0.10.2 toolz 0.12.1 torch 2.0.1 torchaudio 2.0.2 torchvision 0.15.2 tornado 6.4.1 tqdm 4.66.4 transformers 4.41.2 triton 2.0.0 typing_extensions 4.11.0 tzdata 2024.1 urllib3 2.2.2 watchdog 4.0.1 wheel 0.43.0 zipp 3.19.2

[ucasiggcas]

上述代码源于md，https://modelscope.cn/models/ECNU-CILab/ExVideo-SVD-128f-v1

from diffsynth import save_video, ModelManager, SVDVideoPipeline
import torch, requests
from PIL import Image

# Load models
model_manager = ModelManager(torch_dtype=torch.float16, device="cuda",
                             model_id_list=["stable-video-diffusion-img2vid-xt", "ExVideo-SVD-128f-v1"])
pipe = SVDVideoPipeline.from_model_manager(model_manager)

# Generate a video
torch.manual_seed(0)
image = Image.open(requests.get("https://www.modelscope.cn/api/v1/studio/ECNU-CILab/ExVideo-SVD-128f-v1/repo?Revision=master&FilePath=images%2F0.png", stream=True).raw)
image.save("image.png")
video = pipe(
    input_image=image.resize((512, 512)),
    num_frames=128, fps=30, height=512, width=512,
    motion_bucket_id=127,
    num_inference_steps=50,
    min_cfg_scale=2, max_cfg_scale=2, contrast_enhance_scale=1.2
)
save_video(video, "video.mp4", fps=30)

[ucasiggcas]

符合requirements中的 版本要求， torch>=2.0.0 cupy-cuda12x pip transformers controlnet-aux==0.0.7 streamlit streamlit-drawable-canvas imageio imageio[ffmpeg] safetensors einops sentencepiece

[Artiprocher]

非常抱歉，我暂时无法根据您提供的信息定位问题，我们会在后续的工作中尝试复现，可能需要一段时间

[aland68]

test

[aland68]

test

[d8ahazard]

非常抱歉，我暂时无法根据您提供的信息定位问题，我们会在后续的工作中尝试复现，可能需要一段时间

The issue is in the _gaussian function.

Need to change this line:

        # Convert sigma to float32 before performing the exponentiation

to

# Convert sigma to float32 before performing the exponentiation
        sigma = sigma.to(torch.float32)
        x = x.to(torch.float32)

[ucasiggcas]

@Artiprocher 升级torch后解决了这个问题，但是img2video速度好慢啊。 will close the issue，thx