How to avoid the long time required for the first warm up after compile the model

chengzeyi / stable-fast

Best inference performance optimization framework for HuggingFace Diffusers on NVIDIA GPUs.

MIT License

1.06k stars 60 forks source link

How to avoid the long time required for the first warm up after compile the model #98

Closed wz0424 closed 6 months ago

wz0424 commented 6 months ago

The acceleration of sfast performs very well, but the first warm up takes a bit too long. If I need to switch between different models frequently, it will take a very long time on warm up, is there any way to avoid this problem?

Hap-Zhang commented 6 months ago

the same issue +1

chengzeyi commented 6 months ago

@wz0424 @Hap-Zhang See https://github.com/chengzeyi/stable-fast/blob/main/doc/troubleshooting.md#compilation-is-so-slow-how-to-improve-it