[BUG in Stable Diffusion inference] There's an error on CUDAGraph when using deepspeed inference. How to fix it?

microsoft / DeepSpeedExamples

Example models using DeepSpeed

Apache License 2.0

5.83k stars 990 forks source link

Open foin6 opened 4 months ago

foin6 commented 4 months ago

【The Command used to run】: deepspeed --num_gpus 1 test-stable-diffusion.py

【Envs】:

RTX 3090
deepspeed 0.12.6
torch 1.13.1
diffusers 0.26.1
triton 2.0.0.dev20221202

【Traceback information】:

It seems like the CUDAGraph is not available, which confused me a lot. Please take time to review the issue and give me some feedback. Appreciate!

foin6 commented 3 months ago

Would be nice if someone can tell how to fix it.

umchand commented 3 months ago

Can you try upgrading the torch to 2.2.1 and use the latest deepspeed ?